GNU bug report logs - #24116
new snapshot available: diffutils-3.3.50-0353

Previous Next

Package: diffutils;

Reported by: Jim Meyering <jim <at> meyering.net>

Date: Sun, 31 Jul 2016 07:10:01 UTC

Severity: normal

Tags: notabug

Done: Jim Meyering <jim <at> meyering.net>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 24116 in the body.
You can then email your comments to 24116 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-diffutils <at> gnu.org:
bug#24116; Package diffutils. (Sun, 31 Jul 2016 07:10:01 GMT) Full text and rfc822 format available.

Acknowledgement sent to Jim Meyering <jim <at> meyering.net>:
New bug report received and forwarded. Copy sent to bug-diffutils <at> gnu.org. (Sun, 31 Jul 2016 07:10:01 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: bug-diffutils <at> gnu.org
Cc: TP coordinator <coordinator <at> translationproject.org>,
 platform-testers <at> gnu.org
Subject: new snapshot available: diffutils-3.3.50-0353
Date: Sun, 31 Jul 2016 00:08:37 -0700
I released diffutils-3.3 more than three years ago, so this is a bit overdue.
Please give this a workout and report success or failure.
I'd like to make a release within a week or so.

diffutils snapshot:
  http://meyering.net/diff/diffutils-ss.tar.xz      1.3 MB
  http://meyering.net/diff/diffutils-ss.tar.xz.sig
  http://meyering.net/diff/diffutils-3.3.50-0353.tar.xz

NEWS:

** New features

   diff accepts two new options --color and --palette to generate
   and configure colored output.  --color takes an optional argument
   specifying when to colorize a line: --color=always, --color=auto,
   --color=never.  --palette is used to configure which colors are used.

** Bug fixes

  When binary files differ, diff now exits with status 1 as POSIX requires.
  Formerly it exited with status 2.

  Unless the --ignore-file-name-case option is used, diff now
  considers file names to be equal only if they are byte-for-byte
  equivalent.  This fixes a bug where diff in an English locale might
  consider two Asian file names to be the same merely because they
  contain no English characters.

  diff -B no longer generates incorrect output if the two inputs
  each end with a one-byte incomplete line.

  diff --brief no longer reports a difference for unusual identical files.
  For example, when comparing a file like /proc/cmdline (for which the linux
  kernel reports st_size of 0 even though it is not an empty file) to a
  copy of that file's contents residing on a "normal" file system:
    $ f=/proc/cmdline; cp $f k; diff --brief $f k
    Files /proc/cmdline and k differ

** Performance changes

  diff's default algorithm has been adjusted to output higher-quality
  results at somewhat greater computational cost, as CPUs have gotten
  faster since the algorithm was last tweaked in diffutils-2.6 (1993).

=========================================
Changes in diffutils since v3.3:

Andreas Grünbacher (1):
      diff: Improve help text of option --label

Giuseppe Scrivano (5):
      diff: add support for --color
      diff: add --palette
      doc: mention --color and --palette in NEWS
      tests: Add tests for --color and --palette
      diff: --color: fix an infinite recursion bug

Glenn Morris (1):
      doc: fix a reference to emacs' emerge node

Jim Meyering (20):
      maint: post-release administrivia
      doc: mention new option, --no-dereference in 3.3's NEWS
      maint: update bug-reporting address
      maint: update copyright year range in texinfo documentation
      gnulib: update to latest, as well as bootstrap
      gnulib: update to latest
      maint: update copyright year ranges to include 2015; update gnulib
      doc: --no-dereference: improve wording/description
      build: correct man-page generation rule
      build: add gperf to the list of required programs
      tests: arrange to print any skipped-test explanation to tty, too
      diff --brief no longer mistakenly reports diff. with 0-sized /proc/ files
      maint: update copyright year, bootstrap, init.sh
      FIXME: src/diff3: plug a leak
      gnulib: update to latest
      maint: update prereq version of gettext
      maint: don't ignore gitlog-to-changelog failure
      maint: arrange for better URLs in generated announcement message
      maint: arrange for "make distcheck" to work with unreleased automake
      gnulib: update to latest; and tests/init.sh

KO Myung-Hun (2):
      build: double-quote use of $PATH
      diff: skip test if seek is not possible on OS/2 kLIBC

Paul Eggert (20):
      tests: port to Solaris 10 /bin/sh
      diff: tune compare_names_for_qsort
      diff: remove unnecessary decl
      diff: fix bug with Asian file names
      build: update gnulib submodule to latest
      build: update gnulib submodule to latest
      cmp: tune 'cmp a b' for GCC x86
      cmp, diff, sdiff: tune by using rawmemchr
      build: omit -Wsuggest-attribute=pure for lib
      diff: exit with status 1, not 2, when binary files differ
      build: update gnulib submodule to latest
      diff: remove TOO_EXPENSIVE heuristic
      diff: fix bug with -I and overlapping hunks
      diff, sdiff: minor integer overflow fixes
      doc: improve documentation about reading and stdin
      diff: fix two "..." typos in --help output
      diff: fix performance bug with prefix computation
      diff: fix bug with diff -B and incomplete lines
      doc: mention diff -B fix in NEWS
      diff: fix integer overflow problem with --tabsize

Stefano Lattarini (1):
      build: enable 'subdir-objects' automake option


Changes in gnulib since v3.3:

* gnulib db9cad7...348402f (975):
  > autoupdate
  > update from texinfo
  > autoupdate
  > update from texinfo
  > obstack: pacify GCC 6 with -Wnull-dereference
  > update from texinfo
  > doc: mention glibc, OS X, Cygwin [S]SIZE_MAX buglet
  > doc: mention glibc SSIZE_MAX buglet
  > printf-posix: Fix mingw build
  > mktime: call tzset as per POSIX
  > fts: handle readdir() errors
  > intprops: port better to GCC 7
  > intprops-test: port to GCC 6
  > autoupdate
  > update from texinfo
  > autoupdate
  > xalloc-oversized: port to GCC 7; fewer warnings
  > autoupdate (gettext-0.19.8.1)
  > update from texinfo
  > regex: port to Sun C
  > update from texinfo
  > stdbool: Restore __bool_true_false_are_defined check
  > stdbool: Port AC_CHECK_HEADER_STDBOOL to C++11
  > Use GCC_LINT, not lint
  > secure_getenv: Port to many more platforms.
  > update from texinfo
  > canonicalize: Fix broken probe for realpath.
  > unsetenv: relax to LGPLv2+
  > update from texinfo
  > gendocs.sh: Set default TOP_NODE_UP_URL in HTML output.
  > manywarnings: update for GCC 6.1
  > autoupdate
  > intdiv0, memmem, nocrash, strcasestr, strstr: no exit
  > glob: size_t overflow checks
  > glob: don't assume INT_MAX < SIZE_MAX
  > Fix undefined behaviour in gettext.h.
  > autoupdate
  > git-version-gen: avoid undefined shift
  > update from texinfo
  > glob: merge glibc changes into lib/glob.c
  > mktime: spelling fix in comment
  > mktime: port to stricter signed overflow checking
  > mktime: speed up DEBUG_MKTIME benchmarks
  > mktime: resurrect DEBUG_MKTIME testing
  > mktime: simplify DEBUG_MKTIME
  > Port mktime_internal offset to unsigned time_t
  > xstrtol: prohibit monstrosities like "1bB"
  > xstrtod: reinstate setting of *result upon ERANGE
  > xstrtod: modify *result only if no errors
  > btowc: document problems in C locale
  > autoupdate
  > update from texinfo
  > mktime: improve integer overflow checking
  > intprops: check two's complement assumption
  > intprops, mktime, strtol: assume two's complement
  > stdint: port to strict C11 left shift
  > mbrtowc: work around glibc bug#19932
  > update from texinfo
  > autoupdate
  > stdint: detect good enough pre-C++11 stdint.h in C++ mode
  > argp: merge changes from glibc
  > Prefer American spelling for "initialize"
  > autoupdate
  > stddef: support configuring with g++
  > autoupdate
  > autoupdate
  > update from texinfo
  > test-framework-sh: minor cleanups
  > test-framework-sh: revert port to NetBSD 7.0
  > autoupdate
  > Port better to Alpine Linux
  > test-framework-sh: port to NetBSD 7.0
  > update from texinfo
  > gitlog-to-changelog: suppress ignored chatter
  > update from texinfo
  > update from texinfo
  > setlocale: add "sv" to Windows language table
  > update from texinfo
  > sys_select: port to new Cygwin
  > test-userspec.c: do not trigger gcc's new -Wmisleading-indentation
  > time_rz: port to clang -Wunused-const-variable
  > std-gnu11: improve clang support
  > select: port more to Intel 2016.1.150 compiler
  > select: try to port to 2016.1.150 compiler
  > localename-tests: memory allocation fixes
  > intprops: make .h file license match module
  > acl: fix missing return on Cygwin
  > update from texinfo
  > update from texinfo
  > extern-inline: port to PGI CC
  > update from texinfo
  > update from texinfo
  > signbit: port back to pre-C++11 GCC
  > mountlist: recognize autofs-mounted remote file systems, too
  > signbit: port to C++ with GCC 6
  > regex: make it closer to libc
  > regex: merge patches from libc
  > update from texinfo
  > update from texinfo
  > autoupdate
  > autoupdate
  > stdalign: port to older HP and IBM cc
  > stdalign: port to clang 3.7.0
  > update from texinfo
  > readdir_r: now obsolescent
  > Use modern texinfo when syncing install.texi output from autoconf
  > update from texinfo
  > sync install.texi from autoconf
  > misc: port better to gcc -fsanitize=address
  > update from texinfo
  > autoupdate
  > verify-tests: also remove stray test-verify.Tpo
  > std-gnu11: new module
  > update from texinfo
  > update from texinfo
  > get-permissions, strftime: fix grammar in comments
  > gettext: mark as obsolete
  > gnulib-tool: don't give up on ln -s so easily
  > closedir: fix OS/2-related typos
  > openat_proc_name: fix that last '/' is overwritten on OS/2 kLIBC
  > regex: treat [x] as x if x is a unibyte encoding error
  > closedir, dirfd, opendir: port to OpenSolaris 5.10
  > bootstrap: use American spelling
  > @item needed inside @itemize
  > hash-pjw-bare: fix comment
  > wcwidth: Replace also on OpenBSD 5.8
  > update from texinfo
  > gnu-web-doc-update: fix addition of new files
  > utimens-tests: avoid pulling gettext .m4 files
  > regex: pacify static checkers
  > regex: fix [ diagnostic
  > regex: fix memory leaks
  > fts: don't unconditionally use leaf optimization for NFS
  > * gnulib-tool: fix stray debug line in previous patch
  > gnulib-tool: don't assume ln -s works
  > utimes: detect utimes() correctly on OS/2 kLIBC
  > openat_proc_name: port to OS/2 kLIBC
  > stdint: check _INTPTR_T_DECLARED for intptr_t etc.
  > doc: mention unfixed issues with unsupported localtime() values
  > Go back to gettext-0.19.7
  > opendir, closedir, dirfd, fdopendir: port to OS/2 kLIBC
  > dup, dup2, fcntl: support a directory fd on OS/2 kLIBC
  > pipe_filter_ii_execute: port to OS/2 kLIBC
  > wchar: fix "conflicting types" error for __wcwidth on OS/2 kLIBC
  > w32spawn: clear SHELL_SPECIAL_CHARS and SHELL_SPACE_CHAR on OS/2 kLIBC
  > pipe-filter-aux: undefine HAVE_SELECT on KLIBC
  > binary-io: don't put fd in binary mode if it is a console on EMX
  > doc: mention setlocale() issues on OpenBSD
  > sig2str: list all signals on FreeBSD >= 7
  > acl-permissions: port to USE_ACL==0 platforms
  > mktime: rename macro to avoid glibc clash
  > Port "$@" to OpenIndiana ksh93
  > autoupdate
  > Port Universal Time settings to strict POSIX
  > update from texinfo
  > update from texinfo
  > msvc-inval: fix problem with unset shell var
  > autoupdate
  > tests: for compare_(), use cmp -s where available
  > version-etc: new year
  > human: fix output buffer overrun by 1
  > autoupdate
  > maint: add missing ChangeLog entry for previous commit
  > maint: fix operator precedence in mbrtowc test
  > autoupdate
  > regexprops-generic: update from regex.h
  > strftime-tests: avoid false failure on OS X
  > update from texinfo
  > fts: ensure leaf optimization is used for NFS
  > fts: enable leaf optimization for XFS
  > non-recursive-gnulib-prefix-hack: preserve V_GPERF lines
  > intprops: comment fix
  > update from texinfo
  > intprops-test: work around GCC bug 68971
  > autoupdate
  > gnulib-tool: allow multiple --local-dir usage
  > fix freadptr to work with ungetc on all uClibc configs
  > update from texinfo
  > autoupdate
  > parse-datetime: relax license to LGPLv2+, for OSTree
  > update from texinfo
  > autoupdate
  > autoupdate
  > update from texinfo
  > xalloc-oversized: improve performance with GCC 5
  > intprops: new public macro EXPR_SIGNED
  > intprops: fix typo in clang port
  > test-timespec: fix typo in previous change
  > timespec-sub: fix overflow bug; add tests
  > intprops-test: suppress -Woverlength-strings
  > maint: add missing ChangeLog entry for previous commit
  > quotearg: add quotearg_n_style_colon()
  > intprops: revise _WRAPV macros, revert _OVERFLOW
  > intprops: add parentheses for when OP has precedence lower than "-"
  > quotearg: constify get_quoting_style parameters
  > quotearg: add support for $'' shell escaping
  > maint: use a more standard return from mbrtowc test
  > intprops: add WRAPV and const flavors for GCC 5
  > doc: use extended timezone format in iso-8601 example
  > update from texinfo
  > update from texinfo
  > update from texinfo
  > stdalign: port to Sun C 5.9
  > autoupdate
  > update from texinfo
  > autoupdate
  > time_rz: fix comment about tzalloc
  > update from texinfo
  > stdalign: work around pre-4.9 GCC x86 bug
  > maint.mk: sc_tight_scope: remove extraneous expressions
  > time_rz: return NULL if localtime_r fails
  > fts: port to C11 alignof
  > time_rz: avoid warning from bleeding-edge gcc's -Wnonnull
  > maint.mk: _gl_TS_function_match: fix "extern" name extracting regexp
  > maint.mk: sc_tight_scope: factor and support OS X
  > ChangeLog: fix typo: s/cound/count/
  > safe-alloc-tests: fix typo in license header
  > copy-file: fix mem leak in error case
  > localename: control langinfo.h inclusion
  > update from texinfo
  > binary-io, math, pthread, sys_socket, u64, unistd: port to strict C
  > accept4-tests: fix to avoid non portable flags
  > update from texinfo
  > update from texinfo
  > gnulib-tool: fix tests of 'extensions' module
  > unicase/locale-language: fix typo in utf-8 cookie
  > autoupdate
  > xalloc: do not worry about GCC 5 warning on 32 bit
  > xalloc: avoid GCC 5.1 warning on 32 bit
  > uniname/uniname-tests: avoid compiler warnings
  > autoupdate
  > mountlist: clean up of variable duplication
  > c-ctype: do not worry about EBCDIC + char signed
  > c-ctype: port better to z/OS EBCDIC
  > gnulib-common.m4: fix gl_PROG_AR_RANLIB/AM_PROG_AR clash
  > sockets: MS Windows initalization fixes
  > gc: fix detection of installed libgcrypt version
  > c-ctype: rewrite to use inline functions
  > fnmatch: add one more coding cookie
  > maint: add coding cookies to non-ASCII sources
  > gitlog-to-changelog: trim only trailing whitespaces
  > Test that c_iscntrl agrees with iscntrl, etc.
  > c-ctype: improve c_isascii testing
  > Fix ChangeLog typo
  > savewd: remove SAVEWD_CHDIR_READABLE
  > Update ChangeLog to match previous patch.
  > c-ctype: support EBCDIC-style c_isascii
  > c-ctype: assume EBCDIC 1047 for c_iscntrl
  > * modules/c-ctype (Depends-on): Add verify.
  > c-ctype: port better to EBCDIC
  > nanosleep: fix return code for interrupted replacement
  > autoupdate
  > Diagnose ERE '()|\1'
  > Revert previous patch, as it did not fix the bug after all.
  > regex: fix dangling-backreference bug
  > regex: merge patches from libc
  > autoupdate
  > autoupdate
  > autoupdate
  > autoupdate
  > ceill: detect buggy OpenBSD implementation
  > mountlist: add me_mntroot field on Linux machines
  > doc: Describe to use multiple instances of gnulib
  > autoupdate
  > autoupdate
  > base32: mark function as __attribute__ const
  > autoupdate
  > autoupdate
  > gnulib-tool: don't transform binary files with sed
  > autoupdate
  > autoupdate
  > autoupdate
  > autoupdate
  > gperf: respect silent rules
  > Improve port of stdalign to C++11
  > pmccabe2html: fix gawk regex escaping
  > update from texinfo
  > time_rz: port to pedantic memcpy
  > autoupdate
  > time_rz: port better to MinGW
  > time: port __need_time_t to MinGW
  > strftime: fix newly-introduced bug on Solaris
  > test-strftime: test for Solaris bug
  > time_rz: port to Solaris etc.
  > time_rz: now LGPL
  > time_rz: make a constant 'const'
  > time_rz: fix off-by-one typo
  > autoupdate
  > fprintftime, strftime: use timezone_t args
  > time_rz: new module
  > flexmember: license is now unlimited
  > autoupdate
  > maint: update copyright paper procedures
  > autoupdate
  > autoupdate
  > localename: fix link error on Illumos
  > unistr/uN-strtok-tests: avoid a trivial leak
  > acl-permissions: Document FreeBSD ACL_TYPE_NFS4 acls
  > acl-permissions: Fix on FreeBSD
  > file-has-acl, acl-permissions: fix some more HP-UX typos
  > file-has-acl, acl-permissions: fix HP-UX typos
  > regex: match current GNU grep behavior
  > set-permissions.c: adjust acl_from_mode's cpp guard
  > autoupdate
  > tests: restrict shells to those that support 'local'
  > fix mistakes in ChangeLog entries
  > unistr/uN-strtok: handle multibyte delimiters
  > update-copyright: fix test failure with perl >= 5.22
  > u{16,32}-strstr-tests: relax timeout condition
  > gnulib-common.m4: change the ARFLAGS default to 'cr'
  > update from texinfo
  > autoupdate
  > selinux-h: avoid double free after *getfilecon()
  > fix pty related tests issues on Windows
  > acl: fix definition of acl_from_mode on FreeBSD
  > readutmp: port to FreeBSD >= 9
  > mgetgroups: port to strict OS X
  > mgetgroups: fix port to strict OS X
  > linkat: fix invalid definition of LINKAT_SYMLINK_NOTSUP on OS X
  > mgetgroups: port to strict OS X
  > mountlist: avoid an unused-label warning on OS X
  > error.c: correct printf-style format: %d -> %u
  > fts: avoid reading beyond the heap allocation
  > savedir: avoid undefined behavior in qsort call
  > userspec: avoid undefined behavior in gettext call
  > update from texinfo
  > gitlog-to-changelog: improve gitmerge.el commits
  > wchar: fix MinGW compilation warnings
  > uniname/uniname-tests: use pristine data files
  > linked-list, linkedhash-list: avoid compiler warnings
  > libunistring: bump minimum version to 0.9.6
  > uniname/uniname: update to Unicode 8.0.0
  > * lib/gen-uni-tables.c: Style fixes.
  > libunistring: update to Unicode 8.0.0
  > gnu-web-doc-update: add --mirror to remove stale files
  > * lib/set-permissions.c: Spelling fix.
  > update from texinfo
  > acl-permissions: pacify -Wsuggest-attribute=const
  > fsync: document AIX misbehavior
  > acl-permissions: more porting to AIX
  > stdio: Don't redefine gets when using C++
  > acl-permissions: add a cast to port to older AIX
  > acl-permissions: port to older AIX, C89 HP-UX
  > update from texinfo
  > vasprintf-posix: avoid compiling vasnprintf where possible
  > file-has-acl: fix build on Mac OS X 10
  > prefix-gnulib-mk: remove no longer needed special case
  > acl: don't depend on the deprecated qacl module
  > gnulib-tool: concatenate lib_SOURCES to a single line
  > pthread_sigmask: discount system version if a simple macro
  > readlinkat: avoid OS X 10.10 trailing slash bug
  > acl-permissions: Fix build on Mac OS X and older AIX (Bug#20681)
  > acl-permissions: Fix build on Solaris and Cygwin
  > linkat: avoid OS X 10.10 trailing slash with symlink bug
  > unlinkat: handle ignoring of ".." on Darwin 14
  > qacl: split into qcopy-acl and qset-acl
  > gitlog-to-changelog: new option --ignore-line
  > qacl: Reimplement qset_acl and qcopy_acl (Bug#20666)
  > file-has-acl: Split feature tests again (Bug#20667)
  > string: fix build failure on BSD/OSX with FORTIFY_SOURCE
  > stdio: limit __gnu_printf__ witness to gcc 4.4+
  > error: use correct printf attributes on mingw
  > update from texinfo
  > inttypes: force correct mingw PRIdMAX even without <stdio.h>
  > stdio: fix probe on mingw under gcc 5.1
  > autoupdate
  > gitlog-to-changelog: parse "Tiny-change"
  > update from texinfo
  > doc: document glibc posix_fallocate() issues
  > gendocs.sh: document new htmlarg default
  > extern-inline: no need for workaround in GCC 5.1
  > update from texinfo
  > eealloc, pagealign_alloc, xalloc: avoid clang warnings
  > tests: pacify GCC 5.1's stricter printf checking
  > fts: port to GCC 5.1 with --enable-gcc-warnings
  > file-has-acl: port to CentOS 6
  > file-has-acl: always return false when ACLs aren't supported
  > gettext: propagate po/Makefile.in.in too
  > file-has-acl: new module, split from acl
  > manywarnings: add GCC 5.1 warnings
  > autoupdate
  > doc: update FDL template to match FDL examples.
  > lstat: fix cross-compilation 'ln -s' problem
  > gendocs.sh: default to a common CSS style sheet for HTML output
  > gnulib-tool: output bold attribute more portably
  > qacl: Simplify HP-UX acl_nontrivial check
  > acl: On Linux, check for acls without libacl
  > acl, qacl: split off shared functions into separate object file
  > git-version-gen: revert "detect untagged revisions"
  > tempname: avoid unused parameter warnings
  > git-version-gen: detect untagged revisions
  > fseeko: fix build failure on NetBSD >= 6
  > gitlog-to-changelog: port to MS-Windows
  > gendocs: new option --tex for texi2dvi options
  > sync gettext .m4 files from gettext
  > uniname/uniname-tests: fix failure due to alias
  > hash: remove deprecated hash_insert0 function
  > mountlist: remove dependency on libmount
  > stddef: port to pre-C11 GCC on x86
  > quotearg-simple-tests: add missing gl_FUNC_MMAP_ANON dependency
  > yesno: make EOL optional in ENABLE_NLS case also
  > fdopendir-tests: test it does not close its arg
  > gitlog-to-changelog: trim trailing white space
  > gitlog-to-changelog: new option --ignore-matching
  > fdopendir: port better to MinGW
  > fdopendir: fix typo in comment
  > error: document all entry points provided
  > autoupdate
  > autoupdate
  > autoupdate
  > vasnprintf: pacify clang 3.5.0
  > glob, etc.: port to MSVC v18 on MS-Windows 8.1
  > poll: port to MSVC v18 on MS-Windows 8.1
  > tests: support stderr verification with returns_()
  > passfd: avoid valgrind uninitalised data warning
  > uniwbrk/u32-wordbreaks-tests: fix copyright
  > dup2: doc and test for Android bug
  > Replace dup2() on Android
  > Android doesn't define RLIM_SAVED_*
  > autoupdate
  > vasnprintf-posix-tests: use consistent test
  > printf, isinf, etc.: noncanonical != NaN
  > fts: remove redundant close() (trivial)
  > getdtablesize: port better for Android
  > poll: fixes for large fds
  > fcntl: Fix cross compiling
  > dup2, fcntl: cross-compiler better for Android
  > getopt: give accurate ambiguity diagnostic on mem exhaustion
  > getopt: don't crash on memory exhaustion
  > tempname: allow compilation with C++ (trivial)
  > dup2, fcntl: port to AIX
  > getdtablesize, dup2, fcntl: port to Android
  > localename: Implement gl_locale_name_thread_unsafe for Android
  > More changelog fixes
  > fts: avoid crash when a cycle is added while traversing
  > uniname/uniname: support character alias
  > Fix ChangeLog botches in my recent commits
  > duplocale: Fix Android build of duplocale-tests
  > signal_h: Fix Android build
  > net_if: Handle content-free <net/if.h> system headers
  > linkat_nofollow: Add fallback case for cross compiling
  > euidaccess: Fix Android build
  > getugroups: Fix Android build
  > localename: Fix Android build
  > getdtablesize: Fix Android build
  > Assume unbroken ungetc() on Android
  > Fix FILE struct compatibility with Android API level >= 21
  > tests: avoid recent -Werror=unused-variable regression in test-locale
  > maint: various whitespace cleanups in tempname
  > tests: provide returns_() to simplify exit status checking
  > mountlist: only use libmount when specified
  > uniname/unimame-tests: don't link with -lunistring
  > fstrcmp: don't assume strlen < INT_MAX
  > diffseq: prefer ptrdiff_t to ssize_t
  > xalloc: fix typo that suppressed warnings
  > full-read: fix license notice typo
  > crypto/gc: fix a -Wswitch warning
  > autoupdate
  > update from texinfo
  > tempname: new try_tempname function
  > gnulib-tool: fix handling of patch(1) diagnostics
  > bootstrap: exit immediately upon gnulib-tool failure
  > symlinkat: include all required header files
  > update from texinfo
  > update from texinfo
  > update from texinfo
  > localename: support Solaris 12 and illumos
  > locale: fix tests on illumos
  > unictype: avoid undefined left-shift behavior
  > libunistring: bump version of unitypes dependants
  > unictype/category-none: fix link with libunistring
  > unitypes: fix build with installed libunistring
  > time: port to MinGW32 3.21
  > update-copyright: apply to self
  > libunistring: update to Unicode 7.0.0
  > libunistring: update to Unicode 6.3.0
  > libunistring: update to Unicode 6.2.0
  > libunistring: update to Unicode 6.1.0
  > uniwbrk/u32-wordbreaks-tests: add conformance test
  > uniwbrk: ignore Extended/Format characters at BOL
  > test-strstr.c: avoid a trivial leak
  > update-copyright: recognize groff's \(co marker
  > maint.mk: fix compatibility with OS X nm
  > localcharset: improve charset detection on OS/2
  > count-leading-zeros: use 64-bit intrinsics on 32-bit Windows
  > count-leading-zeros: fix pragma typos
  > autoupdate
  > count-leading-zeros: avoid 64-bit intrinsics on 32-bit Windows
  > uniname/uniname: update to Unicode 7.0.0
  > doc: update INSTALL from autoconf
  > stdio: fix use of PRIdMAX on modern mingw
  > Fix check for pthreads.h pollution on Mingw64
  > lib-symbol-versions: cache script check
  > maint: fix grammar nits in propername
  > gendocs: Update bug-report address and URLs to Gnulib's.
  > gendocs: Add 'doc/gendocs_template_min'.
  > gendocs: Change maintainer.
  > gendocs: copyright date and version fix
  > derived INSTALL
  > revert copyright updates in slaved files
  > version-etc: new year
  > autoupdate
  > autoupdate
  > xstrtol: ensure errno is reset
  > utimens: fix dependency typo
  > autoupdate
  > docs: mention why libgen.h is bad
  > autoupdate
  > assure: new module
  > autoupdate
  > stdalign: port better to HP compilers
  > stdalign: work around Apple GCC 4.0 bug
  > gendocs.sh and templates now maintained in gnulib
  > getcwd: fix test failure on OS X 10.9
  > parse-datetime: avoid a compiler warning with byacc
  > unicase/locale-language-tests: fix LOCALE_FR test
  > autoupdate
  > update from texinfo
  > stddef: support C11's max_align_t
  > unistd: fix iOS check conditional
  > posixtm: avoid compiler warning in a better way
  > relocatable: support UNIXROOT in relocate() on EMX
  > freopen: workaround freopen() on OS/2 kLIBC
  > get_shared_library_fullname: port to EMX
  > find_executable: port to EMX
  > sched: check struct sched_param in spawn.h as well
  > support GNU format printf and scanf on mingw
  > bootstrap: Allow perl modules in $buildreq
  > apply _GL_ATTRIBUTE_PURE to some inline functions
  > vasnprintf: fix potential incorrect errno
  > vasnprintf: fix potential use after free
  > autoupdate
  > filevercmp, posixtm: avoid compiler warnings with -O3
  > Fix LDBL80_WORDS macro on big endian platforms.
  > autoupdate
  > git-version-gen: do not print new line characters
  > gnulib-tool: recognize x:* as an absolute path
  > argp: avoid extraneous translation and mem leak with empty pre doc
  > autoupdate
  > doc: mention that _BSD_SOURCE is deprecated for _DEFAULT_SOURCE
  > uniname/uniname-tests: skip if system's libunistring is used
  > printf: fix configure check on big endian systems
  > pipe-filter-gi, pipe-filter-ii: port to AIX
  > gitlog-to-changelog: add --until
  > update from texinfo
  > extern-inline: update commentary about GCC bugs
  > gen-uni-tables: untabify
  > gen-uni-tables: check out-of-range values added to 3-level tables
  > gen-uni-tables: utilize 'assert'
  > gen-uni-tables: cosmetic improvements
  > fcntl-h-tests: port to PA-RISC GNU/Linux
  > fts: port to C89
  > unistd: port to iOS
  > obstack: do not reject malloc-style obstack_chunkfun, obstack_freefun
  > autoupdate
  > update from texinfo
  > obstack: avoid potentially-nonportable function casts
  > obstack: fix macro return values
  > obstack: do not assume system-supplied obstack is size_t safe
  > obstack: port to platforms that #define __alignof__
  > linkat: don't unconditionally replace on GNU/Linux
  > linkat: wrap to handle symlinks on OS X 10.10
  > open, openat: document nonstandard FreeBSD, NetBSD O_NOFOLLOW errno
  > obstack: add NEWS entry for recent incompatible changes
  > mountlist: don't use libmount to decide on dummy/remote
  > maint: add missing ChangeLog entries for Modra's obstack changes
  > obstack: prefer __alignof__ to alignof
  > obstack: prefer alignof to calculating alignments by hand
  > obstack: use size_t alignments and check for overflow
  > obstack: 64-bit obstack support, part 3
  > obstack: 64-bit obstack support, part 2
  > obstack: 64-bit obstack support, part 1
  > obstack: tidy part 2
  > obstack: tidy part 1
  > socketlib, sockets, sys_socket: Use AC_REQUIRE to pacify autoconf.
  > iconv: avoid false detection of non-working iconv
  > bootstrap: print more diagnostics for missing programs
  > bootstrap: only update the gnulib submodule
  > symlinkat: port to AIX 7.1
  > readlinkat: port to AIX 7.1
  > remove spurious {
  > modules/fcntl: fix error reporting by dupfd
  > basename, dirname: Improve documentation.
  > exclude: declare exclude_patopts static
  > autoupdate
  > dirname: support compilation with C++
  > qsort_r: include <config.h>
  > avltree-list: avoid compiler warnings
  > qsort_r: new module, for GNU-style qsort_r
  > strerror_r-posix: support compilation with C++
  > fcntl-h: fix compilation with Intel C++ compiler
  > autoupdate
  > mountlist: use /proc/self/mountinfo when available
  > users.txt: add cmogstored
  > gnulib-tool: Sync with build-aux/bootstrap options
  > gnulib-tool: Fallback to wget when rsync fails
  > maintainer-makefile: add syntax check for useless ';;'
  > pthread, pthread_sigmask, threadlib: port to Ubuntu 14.04
  > error: drop spurious semicolon
  > gnulib-common.m4: port to GCC 4.2.1 and Sun Studio 12 C++
  > manywarnings: add GCC 4.9 warnings
  > vasnprintf: fix bugs in width computation
  > vasnprintf: Avoid signed/unsigned comparison warning.
  > parse-datetime: Avoid signed/unsigned comparison warning
  > qsort_r: new module, for GNU-style qsort_r
  > vla: new module
  > localename: make gl_locale_name_thread really thread-safe on Windows
  > getpass: don't assume struct termios
  > getdtablesize: fall back on sysconf (_SC_OPEN_MAX)
  > vararrays: modernize AC_C_VARARRAYS for C11
  > relocatable-prog-wrapper: port gettext to OS X 10.8 + GCC 4.8.1
  > sys_select: fix FD_ZERO problem on Solaris 10
  > accept: document Solaris 10 type glitch
  > extern-inline: port to FreeBSD, DragonFly
  > autoupdate
  > Use consistent style to check DEBUG macro in regex_internal.c
  > openat-die: use _Noreturn markup
  > test-open: port to cygwin, which lacks Fortify
  > localename: Enforce declarations before statements.
  > test-userspec: don't look up numeric user names
  > localcharset, localename: MS-Windows support for non-default locales
  > announce-gen: avoid failure when Digest::SHA is installed
  > gettext: revert "update macros to version 0.19"
  > regex: don't deref NULL upon heap allocation failure
  > maint.mk: give projects more flexibilty in set_prog_name arguments
  > regex: fix memory leak in compiler
  > announce-gen: avoid perl warnings
  > localename: avoid -Wsuggest-attribute={const,pure} warnings
  > nl_langinfo: Fix last change.
  > Define macros for glibc
  > Sync up error.c with glibc
  > nl_langinfo: fix build under mingw
  > mountlist: do not classify a bind-mounted dir entry as "dummy"
  > maint.mk: less syntax-check noise when SIGPIPE is ignored
  > nl_langinfo: CODESET on MS-Windows and more items from localeconv
  > Bruno Haible has stepped down as maintainer.
  > mktime: merge #if/#ifdef usage from glibc
  > git-version-gen: improve option descriptions
  > regex: fix memory leak in compiler
  > regex: merge patch from libc
  > acl: port to gcc -Wredundant-decls
  > parse-duration: eliminate 68-year duration limit
  > pthread: don't assume AC_CANONICAL_HOST, port better to Solaris, etc.
  > pthread: define thread-safe macros on some platforms
  > regex: don't be multithreaded if USE_UNLOCKED_IO.
  > gettext: update macros to version 0.19
  > select,poll: fix console handle check on windows 8
  > select: fix waiting on anonymous pipes on MS-Windows
  > times: fix to return non constant value on MS-Windows
  > isatty: fix to work on windows 8
  > maint: fix typo in fdl.texi
  > mountlist: avoid hasmntopt const type warning on solaris
  > maintainer-makefile: delete obsolete code
  > maintainer-makefile: avoid spurious error messages
  > rename: avoid unused-but-set-variable compiler warning
  > maint: add ChangeLog entry missing in previous commit
  > rename: mark a label as potentially unused
  > gnulib-common.m4: Fix typo in _GL_UNUSED_LABEL.
  > acl: apply pure attribute to two functions
  > gnulib-common.m4: add _GL_UNUSED_LABEL
  > dup2, fcntl, fcntl-h: port to AIX 7.1
  > printf, config.rpath: Port to FreeBSD 10.
  > ftoastr: work around compiler bug in IBM xlc 12.1
  > valgrind-tests: fixed misleading help message
  > isfinite, isinf, isnan tests: fix for little-endian PowerPC
  > exclude-tests: port to AIX 7.1
  > pthread_sigmask, timer-time: use gl_THREADLIB only if needed
  > gnulib-tool: wget translations using --no-verbose rather than --quiet
  > gnulib-tool: adjust translation wget to avoid a https redirection
  > getlogin_r-tests: check return value rather than errno
  > getlogin_r-tests: fix various issues in recent change
  > fchdir: port 'open' and 'close' redefinitions to AIX 7.1
  > update from texinfo
  > xalloc: don't potentially generate invalid code for xmemdup calls
  > getlogin_r-tests: avoid false failure under sudo/ssh etc.
  > getlogin-tests: avoid false failure under cron
  > mbrtowc.m4: fix a comment typo
  > mbrlen, mbrtowc: fix bug with empty input
  > doc: document mbrtowc and mbrlen problem with empty input
  > doc: document exec* = spawn+exit bug with non-Cygwin Windows platforms
  > autoupdate
  > getlogin-tests: avoid false failure under sudo/ssh etc.
  > mbsstr, quotearg, xstrtol: pacify IRIX 6.5 cc
  > update from texinfo
  > autoupdate
  > autoupdate
  > autoupdate
  > gitlog-to-changelog: revert inclusion of git-log-fix file
  > maint.mk: Relax the copyright check to cater for non FSF projects
  > physmem: use sysinfo if _SC_PHYS_PAGES unavailable
  > exclude: port to strict C99
  > regex: do not depend on malloc-gnu
  > autoupdate
  > expl: avoid incorrect expl(small_value) on OpenBSD 5.4
  > xalloc: allow x2nrealloc (P, PN, S) where P && !*PN
  > fts: avoid unnecessary strlen calls
  > fts: avoid unnecessary strlen calls
  > fts: avoid unnecessary strlen calls
  > autoupdate
  > autoupdate
  > obstack: Remove ancient NeXTSTEP gcc support conditional
  > obstack: merge with glibc changes
  > strftime: wrap macros in "do {...} while(0)"
  > modechange: avoid memory leaks for invalid octal modes
  > autoupdate
  > gitlog-to-changelog: include a dummy git-log-fix file
  > autoupdate
  > update from texinfo
  > gitlog-to-changelog: also include the file, git-log-fix
  > autoupdate
  > regex: port to OS X 10.8.5 en_US.UTF-8 locale
  > maint: fix ChangeLog to match commit record
  > stdint, read-file: fix missing SIZE_MAX on Android (tiny change)
  > parse-datetime: fix crash or infloop in TZ="" parsing
  > * NEWS: Recent changes are not that important.
  > savedir: new symbol for fast-read version
  > unistd: port readlink to Mac OS X 10.3.9
  > * NEWS: Document recent change to diffseq.
  > diffseq: remove TOO_EXPENSIVE heuristic
  > savedir: simplify by using stpcpy
  > spawn: fix link error on uclibc
  > m4: fix gl_TIMER_TIME() detection of threads on uClibc
  > maintainer-makefiles: provide AC_PROG_SED for older autoconf
  > exclude: add support for posix regexps
  > maintainer-makefiles: use $(SED) for syntax check
  > update from texinfo
  > savedir: add sorting arg to savedir, streamsavedir; remove fdsavedir
  > autoupdate
  > update from texinfo
  > update from texinfo
  > file-type: add support for doors and other less-common file types
  > update from texinfo
  > autoupdate
  > pthread: work around winpthread header pollution on mingw
  > qacl: check for fchmod
  > Fix typo in ChangeLog.
  > fdopen-tests: port to Tru64
  > stdalign: port to HP-UX compilers
  > strtoimax: port to platforms lacking 'long long'
  > update from texinfo
  > relocatable-maint.texi: escape braces
  > maint: remove duplicate ChangeLog entry
  > maint.mk: adapt openat.h-include-without-use test
  > relocatable-perl: like relocatable-script, but for Perl scripts
  > relocatable-shell: Update suggested usage in maintainer documentation.
  > tests: fix export bug in previous patch
  > tests: simplify porting to Solaris 10 /bin/sh
  > update from texinfo
  > md5, sha1, sha256, sha512: support older autoconf
  > include_next: port to autoconf 2.63
  > maint: add a gnulib-local rule to keep non-ascii out of .texi files
  > freadable, fwritable, fwriting: declare with the "pure" attribute
  > maint.mk: adapt openat.h-include-without-use test
  > doc: use ASCII in .texi files where UTF-8 isn't needed
  > freading: declare with attribute "pure"
  > manywarnings: remove -Wmudflap
  > autoupdate
  > relocatable-script: remove unused code
  > maint: fix public-submodule-commit to work with newer git
  > autoupdate
  > doc: update main copyright year
  > maint: update copyright
  > version-etc: new year
  > autoupdate
  > passfd: give nicer error for recvfd at eof
  > autoupdate
  > gettimeofday: port recent C++ fix to Emacs
  > gettimeofday: fix C++ crosscompilation
  > qacl: port to Windows better
  > md5, sha1, sha256, sha512: fix compile error in c++ mode
  > open-tests: fix build failure with -Werror=old-style-declaration
  > md5, sha1, sha256, sha512: fix link error with partial libcrypto
  > md5, sha1, sha256, sha512: add gl_SET_CRYPTO_CHECK_DEFAULT
  > md5, sha1, sha256, sha512: add 'auto', and a way to specify default
  > autoupdate
  > bootstrap: warn about BSD toolchain snafu
  > open-tests: port to glibc with _FORTIFY_SOURCE and -O1
  > include_next: minimize code duplication
  > getcwd: fix compile error in configure check
  > regex: suppress core dumps from detection code
  > md5, sha1, sha256, sha512: support mandating use of openssl
  > test-xvasprintf: disable some -Wformat-security diagnostics
  > regex: avoid glibc deadlock during configure
  > md5, sha1, sha256, sha512: use openssl routines if available
  > base64: fix recent compilation regression on some compilers
  > autoupdate
  > autoupdate
  > ignore-value: revert previous code change
  > selinux-h: improve stub types and add more stub functions
  > ignore-value: prefer GCC version back through 2.0
  > pty: Activate the signature wrapper of forkpty.
  > maint: fix gendocs.sh permissions
  > autoupdate
  > quotearg: don't attempt to store 1 << 31 into an "int"
  > error: depend on stdio
  > doc: Improve wording in relocatable-maint.texi.
  > Fix indentation in previous patch.
  > * lib/getgroups.c (posix_getgroups, getgroups) [__APPLE__]:
  > base64: provide a fast path for encoding well sized buffers
  > extern-inline: port better to OS X 10.9
  > fpending: fix typo
  > fpending: fix regression on DragonFly BSD
  > hash: relax license to LGPLv2+, for libguestfs
  > intprops: port to Oracle Studio c99
  > obstack: pacify HP C
  > maint.mk: prefer gpgv2 over gpgv
  > isnan: port to VAX
  > gnulib-tool: protect against CDPATH
  > maint.mk: restore functionality removed by recent change
  > Revert "error: make the module depend on vfprintf-posix"
  > maint.mk: fix "release" target to build _version
  > maint.mk: get current gnulib revision correctly.
  > install-reloc: Support multi-binary installation.
  > selinux-h: really build without selinux when library is missing
  > regex: also remove dependency on HAVE_WCSCOLL
  > xfreopen: fix typo: s/frepoen/freopen/
  > regex: don't depend on wcscoll
  > error: add the printf attribute to a static function
  > error: make the module depend on vfprintf-posix
  > fpending, obstack, strerror-override: use pure+const function attrs
  > extern-inline: make safe for -Wundef usage
  > mkfifo-tests, etc.: allow HP-UX 11.11 bug
  > acl: allow cross-compilation to Gentoo
  > mgetgroups: remove dependency on realloc-gnu
  > regex-tests: port to HP-UX 11.11
  > verify: document some 'assume' pitfalls
  > strtoumax: fix another typo in previous commit
  > strtoumax: fix typo in previous commit.
  > strtoumax: port to Solaris 8
  > autoupdate
  > strtoimax, strtoumax: port to HP-UX 11.11
  > strtoimax: port to HP-UX 11.11
  > New module 'count-trailing-zeros'.
  > count-leading-zeros: port to MSC; support types wider than 64 bits
  > count-one-bits: port to MSC; support types wider than 64 bits
  > mountlist: fix resource leak with MOUNTED_INTERIX_STATVFS
  > tests: improve diagnostic when an assertion fails
  > verify: new macro 'assume'
  > autoupdate
  > dup2, dup3: work around another cygwin crasher
  > getdtablesize: work around cygwin issue
  > pmccabe2html: escaping of special characters
  > manywarnings: enable nicer gcc warning messages
  > timespec: use the new TIMESPEC_RESOLUTION in a few more places
  > warnings: port --enable-gcc-warnings to Solaris Studio 12.3
  > configmake: support new --runstatedir option
  > ctype, string: depend on extern-inline
  > userspec: support optional parameters to parse_user_spec()
  > timespec: new function make_timespec, and new constants
  > stdio: OS X port of putc_unlocked + extern inline
  > signal: OS X port of sigaddset etc. + extern inline
  > extern-inline: do not always suppress extern inline on OS X
  > extern-inline: document fixes for ctype and wctype macros
  > fflush, freadahead, fseeko: Fix for Android
  > pmccabe2html: fix portability issues
  > getgroups: statement without effect
  > autoupdate
  > update from texinfo
  > update from texinfo
  > glob: fix compilation
  > glob: fix build for platforms without __THROW
  > autoupdate
  > regex-quote: fix buffer access out of bounds
  > glob: avoid -Wattribute warnings on glibc
  > headers: check that _GL_INLINE_HEADER_BEGIN is defined
  > bootstrap: remove the --version requirement from ancillary tools
  > gc: support HMAC-SHA256 and HMAC-SHA512.
  > gettext: update to version 0.18.3.1
  > selinux-at: omit unnecessary include
  > autoupdate
  > d-ino: avoid false negative on symlink
  > update from texinfo
  > autoupdate
  > bootstrap: port to OpenBSD sed
  > warnings: minor optimization
  > warnings: check -Wfoo rather than -Wno-foo
  > accept doclicense option
  > accept doclicense option
  > xvasprintf-tests: port to GCC with hardening flags
  > fpending: port to recent Cygwin change to stdio_ext.h
  > no need for tendentious comment
  > regenerate
  > autoupdate
  > sys_time: port to OpenBSD
  > autoupdate
  > bootstrap: support checksum utils without a --status option
  > update from texinfo
  > Fix ChangeLog typo.
  > autoupdate
  > regex: port to non-glibc/lock-using systems
  > update from texinfo
  > autoupdate
  > localecharset: make locale_charset thread-safe on Mac OS X
  > gettext: update to version 0.18.3
  > autoupdate
  > bootstrap: use correct source when copying build-aux files
  > tmpdir: fix bug in VMS port
  > tmpdir: fix typo in previous change
  > * lib/tmpdir.c: Simplify code to add slash; no need for a loop.
  > tmpdir: port to VMS, to // != /, and to long dirs
  > regex: port to --with-included-regex --enable-gcc-warnings non-threaded
  > accept4, dup3, pipe2: port to Cygwin
  > mountlist: add support for deallocating returned list entries
  > stdalign, verify: port to FreeBSD 9.1, to C11, and to C++11
  > fnmatch: don't goto over declaration
  > pipe2: decouple from binary-io a bit
  > mgetgroups: relax license to LGPLv2+
  > xalloc-oversized: relax license to LGPLv2+
  > nproc: relax license to LGPLv2+
  > bootstrap: honor --no-git
  > update from texinfo
  > ignore-value: port to gcc -pedantic
  > extern-inline: port to gcc -std=c89
  > doc: document extern-inline
  > doc: chatter less
  > fflush, fseeko: port to musl cross-compiles
  > msvc-inval: port to mingw-w64
  > getcwd-lgpl: port to Tru64
  > autoupdate
  > tests: port large-fd POSIX spawn tests to OS X
  > autoupdate
  > tests/nap.h: use an adaptive delay to avoid ctime update issues
  > sig2str: port to C++
  > docs: mention cygwin shortcoming in <sys/un.h>
  > vasnprintf: silence mingw compiler warning
  > c-ctype, regex, verify: port to gcc -std=c90 -pedantic
  > regex: adapt to locking regime instead of depending on pthread
  > getgroups: document portability issues
  > test-lchown, test-chown: also skip test if lchown/chown fails with EPERM
  > regex: fix dfa race in multithreaded uses
  > malloca: port to compilers that reject size-zero arrays
  > parse-datetime, tests: don't use "string" + int
  > argmatch: port to C++
  > argp: typo fix
  > autoupdate
  > manywarnings: update for GCC 4.8.0
  > spawn-tests, sys_socket-tests, sys_wait-tests: port to clang
  > getaddrinfo-tests: port --enable-gcc-warnings to clang
  > thread: port --enable-gcc-warnings to clang
  > stdio: use __REDIRECT for fwrite, fwrite_unlocked
  > stdio: port --enable-gcc-warnings to clang
  > sig2str: port --enable-gcc-warnings to clang
  > obstack: port --enable-gcc-warnings to clang
  > memchr2: port --enable-gcc-warnings to clang
  > mbsstr: port --enable-gcc-warnings to clang
  > malloca: port --enable-gcc-warnings to clang
  > inttostr: port --enable-gcc-warnings to clang
  > warnings: port to clang
  > quotearg: do not read beyond end of buffer
  > lock: work around pthread recursive mutexes bug in Mac OS X 10.6
  > mkdir-p: remove assumptions about umask and mode
  > maint.mk: catch more abuse of HAVE_DECL in syntax-check
  > deps: require Automake >= 1.9.6 in generated Makefile fragments
  > autoupdate
  > bootstrap: AC_INIT may have more than four parameters
  > ChangeLog: whitespace cleanup
  > qacl: port to MS-Windows port of GNU Emacs
  > acl: include quote.h
  > fchownat, renameat, unlinkat: update statat dependencies
  > Assume gnulib is checked out from Git, not CVS
  > utimensat-tests, etc.: try to fix some races
  > utimens, utimensat: work around Solaris UTIME_OMIT bug
  > gettext: now it's your responsibility to add -I$(top_builddir)/intl
  > acl: include errno.h to get errno
  > tests: don't assume getdtablesize () <= 10000000
  > extern-inline: work around bug in Sun c99
  > autoupdate
  > qacl: new module, broken out from the acl module
  > alignof, intprops, malloca: port better to IBM's C compiler
  > wctype-h: fix gettext link error on mingw
  > autoupdate
  > regex-tests, regex: allow glibc re_search behavior
  > stdalign: doc fix
  > stdalign: port to stricter ISO C11
  > sys_select, sys_time: port 2013-01-30 Solaris 2.6 fix to Cygwin




Information forwarded to bug-diffutils <at> gnu.org:
bug#24116; Package diffutils. (Sun, 31 Jul 2016 17:19:02 GMT) Full text and rfc822 format available.

Message #8 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Assaf Gordon <assafgordon <at> gmail.com>
To: Jim Meyering <jim <at> meyering.net>
Cc: bug-diffutils <at> gnu.org
Subject: Re: [platform-testers] new snapshot available: diffutils-3.3.50-0353
Date: Sun, 31 Jul 2016 13:17:57 -0400
[Message part 1 (text/plain, inline)]
Hello Jim

> On Jul 31, 2016, at 03:08, Jim Meyering <jim <at> meyering.net> wrote:
> 
> 
> diffutils snapshot:
>  http://meyering.net/diff/diffutils-3.3.50-0353.tar.xz
> 

The "colors" test seems to succeed on Fedora/CentOS/SUSE systems (of various versions), but fail on others (Ubuntu, Debian, FreeBSD, Mac OS X).

Attached are logs from 3 systems. From a cursory look it seems the exact same failure, but I haven't looked deeper.
No other test failures found, but I'll have more results later today.

regards,
 - assaf

[test-suite.darwin14.log.xz (application/octet-stream, attachment)]
[test-suite.freebsd103.log.xz (application/octet-stream, attachment)]
[test-suite.ubuntu1404.log.xz (application/octet-stream, attachment)]
[Message part 5 (text/plain, inline)]


Information forwarded to bug-diffutils <at> gnu.org:
bug#24116; Package diffutils. (Mon, 01 Aug 2016 00:38:01 GMT) Full text and rfc822 format available.

Message #11 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Assaf Gordon <assafgordon <at> gmail.com>
Cc: bug-diffutils <at> gnu.org
Subject: Re: [platform-testers] new snapshot available: diffutils-3.3.50-0353
Date: Sun, 31 Jul 2016 17:36:45 -0700
[Message part 1 (text/plain, inline)]
On Sun, Jul 31, 2016 at 10:17 AM, Assaf Gordon <assafgordon <at> gmail.com> wrote:
> Hello Jim
>
>> On Jul 31, 2016, at 03:08, Jim Meyering <jim <at> meyering.net> wrote:
>>
>> diffutils snapshot:
>>  http://meyering.net/diff/diffutils-3.3.50-0353.tar.xz
>>
>
> The "colors" test seems to succeed on Fedora/CentOS/SUSE systems (of various versions), but fail on others (Ubuntu, Debian, FreeBSD, Mac OS X).
>
> Attached are logs from 3 systems. From a cursory look it seems the exact same failure, but I haven't looked deeper.
> No other test failures found, but I'll have more results later today.

Hi Assaf,
Thank you for all the speedy testing.
I've looked into the failure on a Debian system for which /bin/sh is
dash 0.5.8-2.2.
dash's printf builtin handles \e differently -- that's easy to work
around: use \033, which *is* portable.
More surprising is that this generates no output:

  dash -c 'f() { local t=$(printf '\''\t\t'\''); printf "$t"; }; f'

I.e., piping it into wc -c prints 0.
With bash, it prints the expected pair of TAB bytes.
I found that I could work around this nonsensical behavior by hoisting
the "tab=..." definition up/out of those two functions, or by adding
standard-says-never-necessary double quotes like this:

  dash -c 'f() { local t="$(printf '\''\t\t'\'')"; printf "$t"; }; f'

However, I prefer not to work around it here (and in every other test
script where this comes up), and will insulate all of our test scripts
by rejecting any shell with that misbehavior, so plan to adjust
init.sh to select another shell when it finds this flaw.

On second thought, I will make the local change now, and sleep on the
idea of making init.sh reject dash.
Done in the attached patch.
[esc-tab-dash.diff (text/plain, attachment)]

Information forwarded to bug-diffutils <at> gnu.org:
bug#24116; Package diffutils. (Mon, 01 Aug 2016 01:05:01 GMT) Full text and rfc822 format available.

Message #14 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Assaf Gordon <assafgordon <at> gmail.com>
Cc: bug-diffutils <at> gnu.org
Subject: Re: [platform-testers] new snapshot available: diffutils-3.3.50-0353
Date: Sun, 31 Jul 2016 18:03:50 -0700
[Message part 1 (text/plain, inline)]
By the way, here's another patch that I required, in order to
bootstrap from latest gnulib.
Just pushed, along with the fix+workaround for that test failure.
[remove-gl-lib-reg--c-diff.diff (text/plain, attachment)]

Information forwarded to bug-diffutils <at> gnu.org:
bug#24116; Package diffutils. (Mon, 01 Aug 2016 15:29:01 GMT) Full text and rfc822 format available.

Message #17 received at 24116 <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: Jim Meyering <jim <at> meyering.net>, Assaf Gordon <assafgordon <at> gmail.com>
Cc: 24116 <at> debbugs.gnu.org
Subject: Re: [bug-diffutils] bug#24116: [platform-testers] new snapshot
 available: diffutils-3.3.50-0353
Date: Mon, 1 Aug 2016 09:27:58 -0600
[Message part 1 (text/plain, inline)]
On 07/31/2016 06:36 PM, Jim Meyering wrote:

> dash's printf builtin handles \e differently -- that's easy to work
> around: use \033, which *is* portable.
> More surprising is that this generates no output:
> 
>   dash -c 'f() { local t=$(printf '\''\t\t'\''); printf "$t"; }; f'

That's because you made the classic mistake of using local AND assigning
to the variable at the same time.

This works:

dash -c 'f() { local t; t=$(...); printf "$t"; }; f'

The difference? There is a notion of an assignment context:
http://austingroupbugs.net/view.php?id=351

Bash treats 'local' like 'export' in introducing an assignment context
(so "" around local t=$() is optional); but 'local' is not specified by
POSIX, so dash does NOT treat it as an assignment context.  Since local
is a command name, you MUST use "" in all subsequent words, or else
defer the actual assignments to standalone statements, to get assignment
context.

> 
> I.e., piping it into wc -c prints 0.
> With bash, it prints the expected pair of TAB bytes.
> I found that I could work around this nonsensical behavior by hoisting
> the "tab=..." definition up/out of those two functions, or by adding
> standard-says-never-necessary double quotes like this:
> 
>   dash -c 'f() { local t="$(printf '\''\t\t'\'')"; printf "$t"; }; f'

Umm, the standard is silent.  local is not defined by the standard.

> 
> However, I prefer not to work around it here (and in every other test
> script where this comes up), and will insulate all of our test scripts
> by rejecting any shell with that misbehavior, so plan to adjust
> init.sh to select another shell when it finds this flaw.

It's not a flaw in dash, but in your expectations.


-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Information forwarded to bug-diffutils <at> gnu.org:
bug#24116; Package diffutils. (Mon, 01 Aug 2016 15:39:02 GMT) Full text and rfc822 format available.

Message #20 received at 24116 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Eric Blake <eblake <at> redhat.com>
Cc: Assaf Gordon <assafgordon <at> gmail.com>, 24116 <at> debbugs.gnu.org
Subject: Re: [bug-diffutils] bug#24116: [platform-testers] new snapshot
 available: diffutils-3.3.50-0353
Date: Mon, 1 Aug 2016 08:37:44 -0700
On Mon, Aug 1, 2016 at 8:27 AM, Eric Blake <eblake <at> redhat.com> wrote:
> On 07/31/2016 06:36 PM, Jim Meyering wrote:
>
>> dash's printf builtin handles \e differently -- that's easy to work
>> around: use \033, which *is* portable.
>> More surprising is that this generates no output:
>>
>>   dash -c 'f() { local t=$(printf '\''\t\t'\''); printf "$t"; }; f'
>
> That's because you made the classic mistake of using local AND assigning
> to the variable at the same time.
>
> This works:
>
> dash -c 'f() { local t; t=$(...); printf "$t"; }; f'
>
> The difference? There is a notion of an assignment context:
> http://austingroupbugs.net/view.php?id=351
>
> Bash treats 'local' like 'export' in introducing an assignment context
> (so "" around local t=$() is optional); but 'local' is not specified by
> POSIX, so dash does NOT treat it as an assignment context.  Since local
> is a command name, you MUST use "" in all subsequent words, or else
> defer the actual assignments to standalone statements, to get assignment
> context.
>
>>
>> I.e., piping it into wc -c prints 0.
>> With bash, it prints the expected pair of TAB bytes.
>> I found that I could work around this nonsensical behavior by hoisting
>> the "tab=..." definition up/out of those two functions, or by adding
>> standard-says-never-necessary double quotes like this:
>>
>>   dash -c 'f() { local t="$(printf '\''\t\t'\'')"; printf "$t"; }; f'
>
> Umm, the standard is silent.  local is not defined by the standard.
>
>>
>> However, I prefer not to work around it here (and in every other test
>> script where this comes up), and will insulate all of our test scripts
>> by rejecting any shell with that misbehavior, so plan to adjust
>> init.sh to select another shell when it finds this flaw.
>
> It's not a flaw in dash, but in your expectations.

Thanks for the explanation.

This is a corner of the non-standard that I prefer not to have to
remember and constantly work around. IMHO, bash and zsh get it right,
and dash chose behavior that makes shell scripting with it harder than
necessary.




Information forwarded to bug-diffutils <at> gnu.org:
bug#24116; Package diffutils. (Fri, 05 Aug 2016 12:48:01 GMT) Full text and rfc822 format available.

Message #23 received at 24116 <at> debbugs.gnu.org (full text, mbox):

From: Dave Gordon <david.s.gordon <at> intel.com>
To: Jim Meyering <jim <at> meyering.net>, Assaf Gordon <assafgordon <at> gmail.com>
Cc: 24116 <at> debbugs.gnu.org, dash <at> vger.kernel.org
Subject: Re: [bug-diffutils] bug#24116: [platform-testers] new snapshot
 available: diffutils-3.3.50-0353
Date: Fri, 5 Aug 2016 13:46:56 +0100
On 01/08/16 01:36, Jim Meyering wrote:
> On Sun, Jul 31, 2016 at 10:17 AM, Assaf Gordon <assafgordon <at> gmail.com> wrote:
>> Hello Jim
>>
>>> On Jul 31, 2016, at 03:08, Jim Meyering <jim <at> meyering.net> wrote:
>>>
>>> diffutils snapshot:
>>>  http://meyering.net/diff/diffutils-3.3.50-0353.tar.xz
>>
>> The "colors" test seems to succeed on Fedora/CentOS/SUSE systems (of various versions), but fail on others (Ubuntu, Debian, FreeBSD, Mac OS X).
>>
>> Attached are logs from 3 systems. From a cursory look it seems the exact same failure, but I haven't looked deeper.
>> No other test failures found, but I'll have more results later today.
>
> Hi Assaf,
> Thank you for all the speedy testing.
> I've looked into the failure on a Debian system for which /bin/sh is
> dash 0.5.8-2.2.
> dash's printf builtin handles \e differently -- that's easy to work
> around: use \033, which *is* portable.
> More surprising is that this generates no output:
>
>   dash -c 'f() { local t=$(printf '\''\t\t'\''); printf "$t"; }; f'
>
> I.e., piping it into wc -c prints 0.
> With bash, it prints the expected pair of TAB bytes.
> I found that I could work around this nonsensical behavior by hoisting
> the "tab=..." definition up/out of those two functions, or by adding
> standard-says-never-necessary double quotes like this:
>
>   dash -c 'f() { local t="$(printf '\''\t\t'\'')"; printf "$t"; }; f'
>
> However, I prefer not to work around it here (and in every other test
> script where this comes up), and will insulate all of our test scripts
> by rejecting any shell with that misbehavior, so plan to adjust
> init.sh to select another shell when it finds this flaw.
>
> On second thought, I will make the local change now, and sleep on the
> idea of making init.sh reject dash.
> Done in the attached patch.

No, that's definitely a dash(1) bug, and quite a serious one. Here's a 
variant that makes it more obvious:

# Define our test string, without too much complicated quoting
$ X='f() { local t=$(printf "abc"); printf "$t"; }; f'
$ bash -c "$X" | hd
00000000  61 62 63                                          |abc|
00000003
$ dash -c "$X" | hd
00000000  61 62 63                                          |abc|
00000003
# As expected, we get the same result from bash(1) and dash(1).

# Now try a different test string:
$ X='f() { local t=$(printf "a\tc"); printf "$t"; }; f'
$ bash -c "$X" | hd
00000000  61 09 63                                          |a.c|
00000003
$ dash -c "$X" | hd
00000000  61                                                |a|
00000001
# Wibble! dash(1) has truncated the string at the TAB :(

# In fact it's worse that that
$ X='f() { local t=$(printf "a\tc=d"); printf "$t+$c"; }; f'
$ bash -c "$X" | hd
00000000  61 09 63 3d 64 2b                                 |a.c=d+|
00000006
$ dash -c "$X" | hd
00000000  61 2b 64                                          |a+d|
00000003

What dash(1) appears to have done is silently take the TAB as
the terminator of the containing double-quoted string, AND of
the containing $() construct, as well as a whitespace, so that
the "c=d" is taken as the next argument to the 'local' builtin.

I suspect this unexpected termination of the inner quoted-string
could be quite exploitable!

.Dave.




Information forwarded to bug-diffutils <at> gnu.org:
bug#24116; Package diffutils. (Fri, 05 Aug 2016 14:10:02 GMT) Full text and rfc822 format available.

Message #26 received at 24116 <at> debbugs.gnu.org (full text, mbox):

From: Dave Gordon <david.s.gordon <at> intel.com>
To: Harald van Dijk <harald <at> gigawatt.nl>
Cc: 24116 <at> debbugs.gnu.org, dash <at> vger.kernel.org
Subject: Re: [bug-diffutils] bug#24116: [platform-testers] new snapshot
 available: diffutils-3.3.50-0353
Date: Fri, 5 Aug 2016 15:09:03 +0100
On 05/08/16 14:13, Harald van Dijk wrote:
> On 5-8-2016 14:46, Dave Gordon wrote:
>> On 01/08/16 01:36, Jim Meyering wrote:
>>> On Sun, Jul 31, 2016 at 10:17 AM, Assaf Gordon <assafgordon <at> gmail.com>
>>> wrote:
>>>> Hello Jim
>>>>
>>>>> On Jul 31, 2016, at 03:08, Jim Meyering <jim <at> meyering.net> wrote:
>>>>>
>>>>> diffutils snapshot:
>>>>>  http://meyering.net/diff/diffutils-3.3.50-0353.tar.xz
>>>>
>>>> The "colors" test seems to succeed on Fedora/CentOS/SUSE systems (of
>>>> various versions), but fail on others (Ubuntu, Debian, FreeBSD, Mac
>>>> OS X).
>>>>
>>>> Attached are logs from 3 systems. From a cursory look it seems the
>>>> exact same failure, but I haven't looked deeper.
>>>> No other test failures found, but I'll have more results later today.
>>>
>>> Hi Assaf,
>>> Thank you for all the speedy testing.
>>> I've looked into the failure on a Debian system for which /bin/sh is
>>> dash 0.5.8-2.2.
>>> dash's printf builtin handles \e differently -- that's easy to work
>>> around: use \033, which *is* portable.
>>> More surprising is that this generates no output:
>>>
>>>   dash -c 'f() { local t=$(printf '\''\t\t'\''); printf "$t"; }; f'
>>>
>>> I.e., piping it into wc -c prints 0.
>>> With bash, it prints the expected pair of TAB bytes.
>>> I found that I could work around this nonsensical behavior by hoisting
>>> the "tab=..." definition up/out of those two functions, or by adding
>>> standard-says-never-necessary double quotes like this:
>>>
>>>   dash -c 'f() { local t="$(printf '\''\t\t'\'')"; printf "$t"; }; f'
>>>
>>> However, I prefer not to work around it here (and in every other test
>>> script where this comes up), and will insulate all of our test scripts
>>> by rejecting any shell with that misbehavior, so plan to adjust
>>> init.sh to select another shell when it finds this flaw.
>>>
>>> On second thought, I will make the local change now, and sleep on the
>>> idea of making init.sh reject dash.
>>> Done in the attached patch.
>>
>> No, that's definitely a dash(1) bug, and quite a serious one. Here's a
>> variant that makes it more obvious:
>>
>> # Define our test string, without too much complicated quoting
>> $ X='f() { local t=$(printf "abc"); printf "$t"; }; f'
>> $ bash -c "$X" | hd
>> 00000000  61 62 63                                          |abc|
>> 00000003
>> $ dash -c "$X" | hd
>> 00000000  61 62 63                                          |abc|
>> 00000003
>> # As expected, we get the same result from bash(1) and dash(1).
>>
>> # Now try a different test string:
>> $ X='f() { local t=$(printf "a\tc"); printf "$t"; }; f'
>> $ bash -c "$X" | hd
>> 00000000  61 09 63                                          |a.c|
>> 00000003
>> $ dash -c "$X" | hd
>> 00000000  61                                                |a|
>> 00000001
>> # Wibble! dash(1) has truncated the string at the TAB :(
>>
>> # In fact it's worse that that
>> $ X='f() { local t=$(printf "a\tc=d"); printf "$t+$c"; }; f'
>> $ bash -c "$X" | hd
>> 00000000  61 09 63 3d 64 2b                                 |a.c=d+|
>> 00000006
>> $ dash -c "$X" | hd
>> 00000000  61 2b 64                                          |a+d|
>> 00000003
>>
>> What dash(1) appears to have done is silently take the TAB as
>> the terminator of the containing double-quoted string, AND of
>> the containing $() construct, as well as a whitespace, so that
>> the "c=d" is taken as the next argument to the 'local' builtin.
>>
>> I suspect this unexpected termination of the inner quoted-string
>> could be quite exploitable!
>
> This gets reported relatively frequently. The local command is
> non-standard but a common extension in shells. In the shells that
> provide it, it gets treated the same, syntax-wise, as the standard
> export command, including in dash.
>
> Unfortunately, POSIX currently requires the export command to not have
> any magic quoting, and any POSIX-conforming shell will make
>
>     a="b c=d"
>     export a=$a
>
> set a to b, and c to d. Not so with bash, but that's because bash simply
> isn't POSIX-conforming, even if invoked as sh.
>
> POSIX will require special quoting rules for the export command in the
> future, similar to what bash does today. When it does, dash is likely to
> change to match that, and the local command will likely be changed to
> work the same way.
>
> Right now, though, since the special quoting behaviour is non-standard,
> this is a bug in the script unless the script is explicitly stated to
> work only with specific shells. If the script is meant to be portable,
> even if only across shells that provide the local command, quoting
> $(...) is the right thing to do.
>
> Alternatively:
>
>     local a
>     a=$(...)
>
> should work too, including in dash. Since a=$(...) is not an argument to
> any command here, since it's the shell syntax that says it's an
> assignment rather than the semantics of a particular command, field
> splitting won't happen here.
>
> Cheers,
> Harald van Dijk

Hi,

thanks for the explanation :) I had devised a few more tests and
realised that dash is applying word-splitting after substitution,
as would be expected for ordinary external commands e.g.

$ X="256 if=foo"
$ dd bs=$X
dd: failed to open ‘foo’: No such file or directory

where one would always expect to write bs="$X" with quotes if one wanted 
to ensure that it was taken as a single parameter and without quotes if 
one wanted it to be broken into multiple words.

It was just a surprise to find this (rather than bash's implicit
quoting) applying to dash builtin commands!

Another variant that does work, this time by escaping rather than 
quoting the TAB, and deferring conversion of '\t' into a TAB until after 
the word-splitting:

$ X='f() { local t=$(printf a\\\\tc=d); printf "$t+$c"; }; f'
$ bash -c "$X" | hd
00000000  61 09 63 3d 64 2b                                 |a.c=d+|
00000006
$ dash -c "$X" | hd
00000000  61 09 63 3d 64 2b                                 |a.c=d+|
00000006

Cheers,
.Dave.




Information forwarded to bug-diffutils <at> gnu.org:
bug#24116; Package diffutils. (Fri, 05 Aug 2016 15:34:03 GMT) Full text and rfc822 format available.

Message #29 received at 24116 <at> debbugs.gnu.org (full text, mbox):

From: Harald van Dijk <harald <at> gigawatt.nl>
To: Dave Gordon <david.s.gordon <at> intel.com>, Jim Meyering <jim <at> meyering.net>,
 Assaf Gordon <assafgordon <at> gmail.com>
Cc: 24116 <at> debbugs.gnu.org, dash <at> vger.kernel.org
Subject: Re: [bug-diffutils] bug#24116: [platform-testers] new snapshot
 available: diffutils-3.3.50-0353
Date: Fri, 5 Aug 2016 15:13:25 +0200
On 5-8-2016 14:46, Dave Gordon wrote:
> On 01/08/16 01:36, Jim Meyering wrote:
>> On Sun, Jul 31, 2016 at 10:17 AM, Assaf Gordon <assafgordon <at> gmail.com>
>> wrote:
>>> Hello Jim
>>>
>>>> On Jul 31, 2016, at 03:08, Jim Meyering <jim <at> meyering.net> wrote:
>>>>
>>>> diffutils snapshot:
>>>>  http://meyering.net/diff/diffutils-3.3.50-0353.tar.xz
>>>
>>> The "colors" test seems to succeed on Fedora/CentOS/SUSE systems (of
>>> various versions), but fail on others (Ubuntu, Debian, FreeBSD, Mac
>>> OS X).
>>>
>>> Attached are logs from 3 systems. From a cursory look it seems the
>>> exact same failure, but I haven't looked deeper.
>>> No other test failures found, but I'll have more results later today.
>>
>> Hi Assaf,
>> Thank you for all the speedy testing.
>> I've looked into the failure on a Debian system for which /bin/sh is
>> dash 0.5.8-2.2.
>> dash's printf builtin handles \e differently -- that's easy to work
>> around: use \033, which *is* portable.
>> More surprising is that this generates no output:
>>
>>   dash -c 'f() { local t=$(printf '\''\t\t'\''); printf "$t"; }; f'
>>
>> I.e., piping it into wc -c prints 0.
>> With bash, it prints the expected pair of TAB bytes.
>> I found that I could work around this nonsensical behavior by hoisting
>> the "tab=..." definition up/out of those two functions, or by adding
>> standard-says-never-necessary double quotes like this:
>>
>>   dash -c 'f() { local t="$(printf '\''\t\t'\'')"; printf "$t"; }; f'
>>
>> However, I prefer not to work around it here (and in every other test
>> script where this comes up), and will insulate all of our test scripts
>> by rejecting any shell with that misbehavior, so plan to adjust
>> init.sh to select another shell when it finds this flaw.
>>
>> On second thought, I will make the local change now, and sleep on the
>> idea of making init.sh reject dash.
>> Done in the attached patch.
>
> No, that's definitely a dash(1) bug, and quite a serious one. Here's a
> variant that makes it more obvious:
>
> # Define our test string, without too much complicated quoting
> $ X='f() { local t=$(printf "abc"); printf "$t"; }; f'
> $ bash -c "$X" | hd
> 00000000  61 62 63                                          |abc|
> 00000003
> $ dash -c "$X" | hd
> 00000000  61 62 63                                          |abc|
> 00000003
> # As expected, we get the same result from bash(1) and dash(1).
>
> # Now try a different test string:
> $ X='f() { local t=$(printf "a\tc"); printf "$t"; }; f'
> $ bash -c "$X" | hd
> 00000000  61 09 63                                          |a.c|
> 00000003
> $ dash -c "$X" | hd
> 00000000  61                                                |a|
> 00000001
> # Wibble! dash(1) has truncated the string at the TAB :(
>
> # In fact it's worse that that
> $ X='f() { local t=$(printf "a\tc=d"); printf "$t+$c"; }; f'
> $ bash -c "$X" | hd
> 00000000  61 09 63 3d 64 2b                                 |a.c=d+|
> 00000006
> $ dash -c "$X" | hd
> 00000000  61 2b 64                                          |a+d|
> 00000003
>
> What dash(1) appears to have done is silently take the TAB as
> the terminator of the containing double-quoted string, AND of
> the containing $() construct, as well as a whitespace, so that
> the "c=d" is taken as the next argument to the 'local' builtin.
>
> I suspect this unexpected termination of the inner quoted-string
> could be quite exploitable!

This gets reported relatively frequently. The local command is 
non-standard but a common extension in shells. In the shells that 
provide it, it gets treated the same, syntax-wise, as the standard 
export command, including in dash.

Unfortunately, POSIX currently requires the export command to not have 
any magic quoting, and any POSIX-conforming shell will make

    a="b c=d"
    export a=$a

set a to b, and c to d. Not so with bash, but that's because bash simply 
isn't POSIX-conforming, even if invoked as sh.

POSIX will require special quoting rules for the export command in the 
future, similar to what bash does today. When it does, dash is likely to 
change to match that, and the local command will likely be changed to 
work the same way.

Right now, though, since the special quoting behaviour is non-standard, 
this is a bug in the script unless the script is explicitly stated to 
work only with specific shells. If the script is meant to be portable, 
even if only across shells that provide the local command, quoting 
$(...) is the right thing to do.

Alternatively:

    local a
    a=$(...)

should work too, including in dash. Since a=$(...) is not an argument to 
any command here, since it's the shell syntax that says it's an 
assignment rather than the semantics of a particular command, field 
splitting won't happen here.

Cheers,
Harald van Dijk




Information forwarded to bug-diffutils <at> gnu.org:
bug#24116; Package diffutils. (Fri, 05 Aug 2016 15:55:01 GMT) Full text and rfc822 format available.

Message #32 received at 24116 <at> debbugs.gnu.org (full text, mbox):

From: Dave Gordon <david.s.gordon <at> intel.com>
To: Jim Meyering <jim <at> meyering.net>
Cc: 24116 <at> debbugs.gnu.org
Subject: Re: [bug-diffutils] bug#24116: bug#24116: [platform-testers] new
 snapshot available: diffutils-3.3.50-0353
Date: Fri, 5 Aug 2016 16:54:17 +0100
On 05/08/16 15:09, Dave Gordon wrote:
> On 05/08/16 14:13, Harald van Dijk wrote:
>> On 5-8-2016 14:46, Dave Gordon wrote:
>>> On 01/08/16 01:36, Jim Meyering wrote:
>>>> On Sun, Jul 31, 2016 at 10:17 AM, Assaf Gordon <assafgordon <at> gmail.com>
>>>> wrote:
>>>>> Hello Jim
>>>>>
>>>>>> On Jul 31, 2016, at 03:08, Jim Meyering <jim <at> meyering.net> wrote:
>>>>>>
>>>>>> diffutils snapshot:
>>>>>>  http://meyering.net/diff/diffutils-3.3.50-0353.tar.xz
>>>>>
>>>>> The "colors" test seems to succeed on Fedora/CentOS/SUSE systems (of
>>>>> various versions), but fail on others (Ubuntu, Debian, FreeBSD, Mac
>>>>> OS X).
>>>>>
>>>>> Attached are logs from 3 systems. From a cursory look it seems the
>>>>> exact same failure, but I haven't looked deeper.
>>>>> No other test failures found, but I'll have more results later today.
>>>>
>>>> Hi Assaf,
>>>> Thank you for all the speedy testing.
>>>> I've looked into the failure on a Debian system for which /bin/sh is
>>>> dash 0.5.8-2.2.
>>>> dash's printf builtin handles \e differently -- that's easy to work
>>>> around: use \033, which *is* portable.
>>>> More surprising is that this generates no output:
>>>>
>>>>   dash -c 'f() { local t=$(printf '\''\t\t'\''); printf "$t"; }; f'
>>>>
>>>> I.e., piping it into wc -c prints 0.
>>>> With bash, it prints the expected pair of TAB bytes.
>>>> I found that I could work around this nonsensical behavior by hoisting
>>>> the "tab=..." definition up/out of those two functions, or by adding
>>>> standard-says-never-necessary double quotes like this:
>>>>
>>>>   dash -c 'f() { local t="$(printf '\''\t\t'\'')"; printf "$t"; }; f'
>>>>
>>>> However, I prefer not to work around it here (and in every other test
>>>> script where this comes up), and will insulate all of our test scripts
>>>> by rejecting any shell with that misbehavior, so plan to adjust
>>>> init.sh to select another shell when it finds this flaw.
>>>>
>>>> On second thought, I will make the local change now, and sleep on the
>>>> idea of making init.sh reject dash.
>>>> Done in the attached patch.

[snip]

>> Alternatively:
>>
>>     local a
>>     a=$(...)
>>
>> should work too, including in dash. Since a=$(...) is not an argument to
>> any command here, since it's the shell syntax that says it's an
>> assignment rather than the semantics of a particular command, field
>> splitting won't happen here.
>>
>> Cheers,
>> Harald van Dijk

Hi,

after Harald's explanation, can I suggest you change the script to
separate the "local" and the assignment? That appears to work on all
shells, including dash(1)

$ dash -c 'f() { local t; t=$(printf '\''\t\t'\''); printf "$t"; }; f' | hd
00000000  09 09                                             |..|
00000002

It's a really minimal (3-character) change, and it's less ugly than
adding the extra quotes that the standard seems to say should never
be needed :)

.Dave.




Information forwarded to bug-diffutils <at> gnu.org:
bug#24116; Package diffutils. (Fri, 05 Aug 2016 16:22:02 GMT) Full text and rfc822 format available.

Message #35 received at 24116 <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: Harald van Dijk <harald <at> gigawatt.nl>,
 Dave Gordon <david.s.gordon <at> intel.com>,
 Jim Meyering <jim <at> meyering.net>, Assaf Gordon <assafgordon <at> gmail.com>
Cc: 24116 <at> debbugs.gnu.org, dash <at> vger.kernel.org
Subject: Re: [bug-diffutils] bug#24116: [platform-testers] new snapshot
 available: diffutils-3.3.50-0353
Date: Fri, 5 Aug 2016 10:21:20 -0600
[Message part 1 (text/plain, inline)]
On 08/05/2016 07:13 AM, Harald van Dijk wrote:

> 
> Unfortunately, POSIX currently requires the export command to not have
> any magic quoting, and any POSIX-conforming shell will make
> 
>     a="b c=d"
>     export a=$a
> 
> set a to b, and c to d. Not so with bash, but that's because bash simply
> isn't POSIX-conforming, even if invoked as sh.

Not quite so fast.

> 
> POSIX will require special quoting rules for the export command in the
> future, similar to what bash does today. When it does, dash is likely to
> change to match that, and the local command will likely be changed to
> work the same way.

POSIX has already issued an interpretation request, and therefore, any
CURRENT implementations (including bash) that already behave as
permitted by the interpretation ARE conforming.  Dash is still currently
conforming per the current version of POSIX, but will be non-conforming
when the interpretation request goes live in the next revision of POSIX,
while bash is already conforming both now and in the future revision,
without change.  So I would suggest that dash make the change to
implement assignment context to 'export', 'readonly', and 'local'
sooner, rather than later.

http://austingroupbugs.net/view.php?id=351

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Information forwarded to bug-diffutils <at> gnu.org:
bug#24116; Package diffutils. (Fri, 05 Aug 2016 17:16:02 GMT) Full text and rfc822 format available.

Message #38 received at 24116 <at> debbugs.gnu.org (full text, mbox):

From: Harald van Dijk <harald <at> gigawatt.nl>
To: Eric Blake <eblake <at> redhat.com>, Dave Gordon <david.s.gordon <at> intel.com>,
 Jim Meyering <jim <at> meyering.net>, Assaf Gordon <assafgordon <at> gmail.com>
Cc: 24116 <at> debbugs.gnu.org, dash <at> vger.kernel.org
Subject: Re: [bug-diffutils] bug#24116: [platform-testers] new snapshot
 available: diffutils-3.3.50-0353
Date: Fri, 5 Aug 2016 19:15:02 +0200
On 05/08/2016 18:21, Eric Blake wrote:
> On 08/05/2016 07:13 AM, Harald van Dijk wrote:
>> Unfortunately, POSIX currently requires the export command to not have
>> any magic quoting, and any POSIX-conforming shell will make
>>
>>     a="b c=d"
>>     export a=$a
>>
>> set a to b, and c to d. Not so with bash, but that's because bash simply
>> isn't POSIX-conforming, even if invoked as sh.
>
> Not quite so fast.
>
>>
>> POSIX will require special quoting rules for the export command in the
>> future, similar to what bash does today. When it does, dash is likely to
>> change to match that, and the local command will likely be changed to
>> work the same way.
>
> POSIX has already issued an interpretation request, and therefore, any
> CURRENT implementations (including bash) that already behave as
> permitted by the interpretation ARE conforming. [...]

> http://austingroupbugs.net/view.php?id=351

An interpretation request doesn't automatically mean that all current 
implementations are conforming, does it? It only means that when the 
interpretation response says so. In this case, the response says that 
the standard does not allow bash/ksh's behaviour.

http://austingroupbugs.net/view.php?id=351#c865:

> Interpretation response
> ------------------------
> The standard states that the current behavior of ksh93 does not conform, and conforming implementations must conform to this. However, concerns have been raised about this which are being referred to the sponsor.

The fact that this is seen as a defect in the standard which will be 
fixed, rather than a defect in bash/ksh, doesn't make bash/ksh conform, 
it merely means that bash/ksh shouldn't be changed to conform.




Information forwarded to bug-diffutils <at> gnu.org:
bug#24116; Package diffutils. (Fri, 05 Aug 2016 20:36:02 GMT) Full text and rfc822 format available.

Message #41 received at 24116 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Dave Gordon <david.s.gordon <at> intel.com>
Cc: 24116 <at> debbugs.gnu.org
Subject: Re: [bug-diffutils] bug#24116: bug#24116: [platform-testers] new
 snapshot available: diffutils-3.3.50-0353
Date: Fri, 5 Aug 2016 13:35:30 -0700
On Fri, Aug 5, 2016 at 8:54 AM, Dave Gordon <david.s.gordon <at> intel.com> wrote:
...
>
> after Harald's explanation, can I suggest you change the script to
> separate the "local" and the assignment? That appears to work on all
> shells, including dash(1)
>
> $ dash -c 'f() { local t; t=$(printf '\''\t\t'\''); printf "$t"; }; f' | hd
> 00000000  09 09                                             |..|
> 00000002
>
> It's a really minimal (3-character) change, and it's less ugly than
> adding the extra quotes that the standard seems to say should never
> be needed :)

Thanks, but I've already fixed it by hoisting the definition to global scope:
http://git.sv.gnu.org/cgit/diffutils.git/commit/?id=55596fcd4846a802d




Added tag(s) notabug. Request was from Jim Meyering <jim <at> meyering.net> to control <at> debbugs.gnu.org. (Thu, 04 May 2017 17:01:02 GMT) Full text and rfc822 format available.

bug closed, send any further explanations to 24116 <at> debbugs.gnu.org and Jim Meyering <jim <at> meyering.net> Request was from Jim Meyering <jim <at> meyering.net> to control <at> debbugs.gnu.org. (Thu, 04 May 2017 17:01:02 GMT) Full text and rfc822 format available.

bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Fri, 02 Jun 2017 11:24:04 GMT) Full text and rfc822 format available.

This bug report was last modified 6 years and 301 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.