GNU bug report logs - #16329
`head --lines=-0' prints nothing if no newline at the EOF

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 16329 in the body.
You can then email your comments to 16329 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Алексей Шилин <rootlexx <at> mail.ru>
To: bug-coreutils <at> gnu.org
Subject: `head --lines=-0' prints nothing if no newline at the EOF
Date: Fri, 03 Jan 2014 19:50:56 +0400

[Message part 1 (text/plain, inline)]

Hi,

If one runs `head --lines=-0 somefile', he'll sometimes get no output instead of full file contents as
expected after reading the manual:

-n, --lines=[-]K
        print the first K lines instead of the first 10; with the lead‐
        ing `-', print all but the last K lines of each file

It depends on whether the file has a trailing newline.

For example:

$ printf '1\n2\n3' > test1
$ printf '4\n5\n' > test2
$ head -n -0 test*
==> test1 <==

==> test2 <==
4
5
$ tail -n +0 test*   # Just for comparison
==> test1 <==
1
2
3
==> test2 <==
4
5
$ 

I'm using Debian GNU/Linux 7 wheezy and coreutils-8.13; I've also tested coreutils-8.22, and it has the
same issue.

A proposed patch fixing the bug is attached. It makes head behave exactly like `tail -n +0' on same files.

-- 
Алексей Шилин

[head_nlines_fix.patch (application/x-patch, attachment)]

Message #8 received at 16329 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Алексей Шилин <rootlexx <at> mail.ru>
Cc: 16329 <at> debbugs.gnu.org
Subject: Re: bug#16329: `head --lines=-0' prints nothing if no newline at
 the EOF
Date: Fri, 03 Jan 2014 16:45:07 +0000

On 01/03/2014 03:50 PM, Алексей Шилин wrote:
> Hi,
> 
> If one runs `head --lines=-0 somefile', he'll sometimes get no output instead of full file contents as
> expected after reading the manual:
> 
> -n, --lines=[-]K
>         print the first K lines instead of the first 10; with the lead‐
>         ing `-', print all but the last K lines of each file
> 
> It depends on whether the file has a trailing newline.
> 
> For example:
> 
> $ printf '1\n2\n3' > test1
> $ printf '4\n5\n' > test2
> $ head -n -0 test*
> ==> test1 <==
> 
> ==> test2 <==
> 4
> 5
> $ tail -n +0 test*   # Just for comparison
> ==> test1 <==
> 1
> 2
> 3
> ==> test2 <==
> 4
> 5
> $ 
> 
> I'm using Debian GNU/Linux 7 wheezy and coreutils-8.13; I've also tested coreutils-8.22, and it has the
> same issue.
> 
> A proposed patch fixing the bug is attached. It makes head behave exactly like `tail -n +0' on same files.
> 

Thanks for the clear reproducer and patch.
I'll probably apply this in your name with a test.

thanks!
Pádraig.

Message #11 received at 16329 <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: Алексей Шилин <rootlexx <at> mail.ru>,
 16329 <at> debbugs.gnu.org
Subject: Re: bug#16329: `head --lines=-0' prints nothing if no newline at
 the EOF
Date: Fri, 03 Jan 2014 09:57:25 -0700

[Message part 1 (text/plain, inline)]

On 01/03/2014 08:50 AM, Алексей Шилин wrote:
> Hi,
> 
> If one runs `head --lines=-0 somefile', he'll sometimes get no output instead of full file contents as
> expected after reading the manual:
> 
> -n, --lines=[-]K
>         print the first K lines instead of the first 10; with the lead‐
>         ing `-', print all but the last K lines of each file
> 
> It depends on whether the file has a trailing newline.

Per POSIX, 'head' is only required to operate on "text files"
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/head.html
and a "text file" must either be empty or have a trailing newline:
http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_397

So this is not necessarily a bug.  That said, since POSIX doesn't
specify what we should do, we are free to make it behave differently.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Message #14 received at 16329 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Eric Blake <eblake <at> redhat.com>
Cc: 16329 <at> debbugs.gnu.org,
 Алексей Шилин <rootlexx <at> mail.ru>
Subject: Re: bug#16329: `head --lines=-0' prints nothing if no newline at
 the EOF
Date: Fri, 03 Jan 2014 17:02:47 +0000

On 01/03/2014 04:57 PM, Eric Blake wrote:
> On 01/03/2014 08:50 AM, Алексей Шилин wrote:
>> Hi,
>>
>> If one runs `head --lines=-0 somefile', he'll sometimes get no output instead of full file contents as
>> expected after reading the manual:
>>
>> -n, --lines=[-]K
>>         print the first K lines instead of the first 10; with the lead‐
>>         ing `-', print all but the last K lines of each file
>>
>> It depends on whether the file has a trailing newline.
> 
> Per POSIX, 'head' is only required to operate on "text files"
> http://pubs.opengroup.org/onlinepubs/9699919799/utilities/head.html
> and a "text file" must either be empty or have a trailing newline:
> http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_397
> 
> So this is not necessarily a bug.  That said, since POSIX doesn't
> specify what we should do, we are free to make it behave differently.

BTW I still don't think POSIX is specifying that text files without
a trailing newline are not text files.

It states that text files can have zero or more lines,
implying that the trailing new line is optional.

thanks,
Pádraig.

Message #17 received at 16329 <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 16329 <at> debbugs.gnu.org,
 Алексей Шилин <rootlexx <at> mail.ru>
Subject: Re: bug#16329: `head --lines=-0' prints nothing if no newline at
 the EOF
Date: Fri, 03 Jan 2014 10:10:09 -0700

[Message part 1 (text/plain, inline)]

On 01/03/2014 10:02 AM, Pádraig Brady wrote:
>>> It depends on whether the file has a trailing newline.
>>
>> Per POSIX, 'head' is only required to operate on "text files"
>> http://pubs.opengroup.org/onlinepubs/9699919799/utilities/head.html
>> and a "text file" must either be empty or have a trailing newline:
>> http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_397
>>
>> So this is not necessarily a bug.  That said, since POSIX doesn't
>> specify what we should do, we are free to make it behave differently.
> 
> BTW I still don't think POSIX is specifying that text files without
> a trailing newline are not text files.
> 
> It states that text files can have zero or more lines,

Where "lines" is also a well-defined POSIX term:
http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_206

> implying that the trailing new line is optional.

Wrong.  The trailing newline character is mandatory in text files.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Message #20 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: bug-coreutils <at> gnu.org
Subject: Re: bug#16329: `head --lines=-0' prints nothing if no newline at
 the EOF
Date: Fri, 03 Jan 2014 09:34:28 -0800

Pádraig Brady wrote:
> It states that text files can have zero or more lines,
> implying that the trailing new line is optional.

I think Eric's right here; if the file is nonempty,
a trailing newline is required.

I'm old enough to remember when text files were not
allowed to be empty (!).  Totally bizarre, but that's
the way POSIX was until 2008.  Before that, every text
file had to end with a newline byte.  See:

http://pubs.opengroup.org/onlinepubs/000095399/basedefs/xbd_chap03.html#tag_03_392

Message #23 received at 16329 <at> debbugs.gnu.org (full text, mbox):

From: Алексей Шилин <rootlexx <at> mail.ru>
To: 16329 <at> debbugs.gnu.org
Subject: Re: bug#16329: Acknowledgement (`head --lines=-0' prints nothing if no newline at the EOF)
Date: Sat, 04 Jan 2014 00:40:27 +0400

[Message part 1 (text/plain, inline)]

Looks like I was overly cautious about decrementing an unsigned...

size_t n = bytes_read;
while (n)
  {
    if (all_lines)
      n -= n ? 1 : 0;        // ...here.
    else

As it is under `while (n)' statement, n is always true here, and thus the ternary operator, though makes no
harm, is needless, and the whole line can be replaced with just `n--;'. Sorry for that.

The fixed version of the original patch is attached.

-- 
Алексей Шилин

[head_nlines_fix2.patch (application/x-patch, attachment)]

Message #26 received at 16329 <at> debbugs.gnu.org (full text, mbox):

From: Алексей Шилин <rootlexx <at> mail.ru>
To: Eric Blake <eblake <at> redhat.com>
Cc: 16329 <at> debbugs.gnu.org
Subject: Re[2]: bug#16329: `head --lines=-0' prints nothing if no newline at the EOF
Date: Sat, 04 Jan 2014 21:52:44 +0400

Пятница,  3 января 2014, 9:57 -07:00 от Eric Blake <eblake <at> redhat.com>:
>Per POSIX, 'head' is only required to operate on "text files"
>http://pubs.opengroup.org/onlinepubs/9699919799/utilities/head.html
>and a "text file" must either be empty or have a trailing newline:
>http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_397
>
>So this is not necessarily a bug.

While I generally agree, that current `head' implementation doesn't violate the standard, I still consider
it a bug, because:

 * `head -n -N filename' works the same way on files both with and without trailing newline for all N
   except N == 0. For example:

   $ printf '1\n2\n3' > test
   $ head -n -1 test
   1
   2
   $ head -n -0 test
   $ 

   In fact, `head' was written explicitly to consider that trailing newline is not always present [1].

   The only reason N == 0 case works differently is that the `n_lines' variable, being unsigned and equal
   to 0, is decremented [2], and, as a result of an integer underflow, gets a very big (UINT64_MAX)
   value, which in turn, having in mind the 'all but the last K lines' meaning, leads to empty output.

   I guess, it's highly unlikely to be the intended behavior.

 * `tail -n +0 filename' has the very same meaning according to its' manual, and it works just fine:

   $ tail -n +0 test; echo
   1
   2
   3
   $ 

To sum up, current `head' behavior is likely unintended, inconsistent and confusing. That's why I still
consider it a bug, maybe of a lesser priority.

[1] http://git.savannah.gnu.org/gitweb/?p=coreutils.git;a=blob;f=src/head.c;h=ddaa9906b41ecc20fbb67f87fab43304c9ccb674;hb=HEAD#l639
[2] http://git.savannah.gnu.org/gitweb/?p=coreutils.git;a=blob;f=src/head.c;h=ddaa9906b41ecc20fbb67f87fab43304c9ccb674;hb=HEAD#l641

-- 
Алексей Шилин

Message #33 received at 16329-done <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Алексей Шилин <rootlexx <at> mail.ru>
Cc: 16329-done <at> debbugs.gnu.org
Subject: Re: bug#16329: Acknowledgement (`head --lines=-0' prints nothing
 if no newline at the EOF)
Date: Wed, 29 Jan 2014 11:55:53 +0000

[Message part 1 (text/plain, inline)]

On 01/03/2014 08:40 PM, Алексей Шилин wrote:
> Looks like I was overly cautious about decrementing an unsigned...
> 
> size_t n = bytes_read;
> while (n)
>   {
>     if (all_lines)
>       n -= n ? 1 : 0;        // ...here.
>     else
> 
> As it is under `while (n)' statement, n is always true here, and thus the ternary operator, though makes no
> harm, is needless, and the whole line can be replaced with just `n--;'. Sorry for that.
> 
> The fixed version of the original patch is attached.
> 

I've updated the patch to also handle the pipe case,
and I added a test. Will push soon.

thanks,
Pádraig.

[head--lines-0.patch (text/x-patch, attachment)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.