GNU bug report logs - #15604
sha256sum (and others of the *sum family) lacks important option

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 15604 in the body.
You can then email your comments to 15604 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Hadmut Danisch <hadmut <at> danisch.de>
To: bug-coreutils <at> gnu.org
Subject: sha256sum (and others of the *sum family) lacks important option
Date: Sun, 13 Oct 2013 12:31:15 +0200

Hi,

the hashsum check files like  sha256sum would be quite usefull (and
improve security), if they could easily be used within shellscripts to
verify downloads. E.g. Ubuntu provides signed hash files like

http://releases.ubuntu.com/precise/SHA256SUMS
http://releases.ubuntu.com/precise/SHA256SUMS.gpg


But one rarely downloads all files, only the needed ones.

Unfortunately, sha256sum prints warnings and exits with exit status 1
when files are missing, even when the present files are correct. This
makes checking the files more difficult in shellscripts and might keep
most script authors from checking downloads.

It would be significantly more usefull and thus more secure if  the *sum
commands had an additional option to not report missing files as an
error and to verify just the present files.

regards
Hadmut

Message #8 received at 15604 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Hadmut Danisch <hadmut <at> danisch.de>
Cc: 15604 <at> debbugs.gnu.org
Subject: Re: bug#15604: sha256sum (and others of the *sum family) lacks
 important option
Date: Mon, 14 Oct 2013 15:28:36 +0100

On 10/13/2013 11:31 AM, Hadmut Danisch wrote:
> Hi,
> 
> the hashsum check files like  sha256sum would be quite usefull (and
> improve security), if they could easily be used within shellscripts to
> verify downloads. E.g. Ubuntu provides signed hash files like
> 
> http://releases.ubuntu.com/precise/SHA256SUMS
> http://releases.ubuntu.com/precise/SHA256SUMS.gpg
> 
> 
> But one rarely downloads all files, only the needed ones.
> 
> Unfortunately, sha256sum prints warnings and exits with exit status 1
> when files are missing, even when the present files are correct. This
> makes checking the files more difficult in shellscripts and might keep
> most script authors from checking downloads.
> 
> It would be significantly more usefull and thus more secure if  the *sum
> commands had an additional option to not report missing files as an
> error and to verify just the present files.

This was requested previously:
http://lists.gnu.org/archive/html/coreutils/2010-12/msg00032.html

The option presented there might work for you? i.e.

  md5sum -c file.sum 2>/dev/null | grep FAILED$

Another disadvantage to the workaround than stated in the original thread,
is that various other md5sum errors might not be diagnosed
(like missing file.sum for example).

So I'm not against a new option for this,
but it warrants discussion.

thanks,
Pádraig.

Message #13 received at 15604-done <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Bernhard Voelker <mail <at> bernhard-voelker.de>, Luther
 <lutheroto <at> gmail.com>, coreutils <at> gnu.org, Kamil Paral <kparal <at> redhat.com>,
 15604-done <at> debbugs.gnu.org
Subject: Re: [coreutils] [PATCH] md5sum: Add option to ignore non-existant
 files
Date: Mon, 23 Nov 2015 13:20:12 +0000

> I'll push a bit later today.

Pushed at http://git.sv.gnu.org/gitweb/?p=coreutils.git;a=commitdiff;h=v8.24-91-g9fd0662
Marking http://bugs.gnu.org/15604 done

Message #16 received at 15604-done <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Pádraig Brady <P <at> draigbrady.com>
Cc: 15604-done <at> debbugs.gnu.org, Bernhard Voelker <mail <at> bernhard-voelker.de>,
 Luther <lutheroto <at> gmail.com>, Coreutils <coreutils <at> gnu.org>,
 Kamil Paral <kparal <at> redhat.com>
Subject: Re: [coreutils] [PATCH] md5sum: Add option to ignore non-existant
 files
Date: Mon, 23 Nov 2015 17:05:44 +0100

On Mon, Nov 23, 2015 at 2:20 PM, Pádraig Brady <P <at> draigbrady.com> wrote:
>> I'll push a bit later today.
>
> Pushed at http://git.sv.gnu.org/gitweb/?p=coreutils.git;a=commitdiff;h=v8.24-91-g9fd0662
> Marking http://bugs.gnu.org/15604 done

Given how this warns/fails when using --check does nothing,

  $ :|sha1sum --check
  sha1sum: 'standard input': no properly formatted SHA1 checksum lines found
  [Exit 1]

should using --check with --ignore-missing also warn/fail when it
verifies no checksum?

  $ :|sha1sum |sed s/-/no-such/ |sha1sum --check --ignore-missing; echo $?
  0

Message #19 received at 15604-done <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Jim Meyering <jim <at> meyering.net>
Cc: 15604-done <at> debbugs.gnu.org, Bernhard Voelker <mail <at> bernhard-voelker.de>,
 Luther <lutheroto <at> gmail.com>, Coreutils <coreutils <at> gnu.org>,
 Kamil Paral <kparal <at> redhat.com>
Subject: Re: [coreutils] [PATCH] md5sum: Add option to ignore non-existant
 files
Date: Mon, 23 Nov 2015 16:24:15 +0000

On 23/11/15 16:05, Jim Meyering wrote:
> On Mon, Nov 23, 2015 at 2:20 PM, Pádraig Brady <P <at> draigbrady.com> wrote:
>>> I'll push a bit later today.
>>
>> Pushed at http://git.sv.gnu.org/gitweb/?p=coreutils.git;a=commitdiff;h=v8.24-91-g9fd0662
>> Marking http://bugs.gnu.org/15604 done
> 
> Given how this warns/fails when using --check does nothing,
> 
>   $ :|sha1sum --check
>   sha1sum: 'standard input': no properly formatted SHA1 checksum lines found
>   [Exit 1]
> 
> should using --check with --ignore-missing also warn/fail when it
> verifies no checksum?
> 
>   $ :|sha1sum |sed s/-/no-such/ |sha1sum --check --ignore-missing; echo $?
>   0

It's a fair point, but I see the first error as verifying the
checksum file itself, and so separate functionality.

Related to this is outputting "MISSING" as well as "OK"
unless --quiet is specified, though I thought the lack
of "OK" if no files found would be enough indication
of an issue in the normal usage?

cheers,
Pádraig

Message #22 received at 15604-done <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Pádraig Brady <P <at> draigbrady.com>
Cc: 15604-done <at> debbugs.gnu.org, Bernhard Voelker <mail <at> bernhard-voelker.de>,
 Luther <lutheroto <at> gmail.com>, Coreutils <coreutils <at> gnu.org>,
 Kamil Paral <kparal <at> redhat.com>
Subject: Re: [coreutils] [PATCH] md5sum: Add option to ignore non-existant
 files
Date: Mon, 23 Nov 2015 17:41:13 +0100

On Mon, Nov 23, 2015 at 5:24 PM, Pádraig Brady <P <at> draigbrady.com> wrote:
> On 23/11/15 16:05, Jim Meyering wrote:
>> On Mon, Nov 23, 2015 at 2:20 PM, Pádraig Brady <P <at> draigbrady.com> wrote:
>>>> I'll push a bit later today.
>>>
>>> Pushed at http://git.sv.gnu.org/gitweb/?p=coreutils.git;a=commitdiff;h=v8.24-91-g9fd0662
>>> Marking http://bugs.gnu.org/15604 done
>>
>> Given how this warns/fails when using --check does nothing,
>>
>>   $ :|sha1sum --check
>>   sha1sum: 'standard input': no properly formatted SHA1 checksum lines found
>>   [Exit 1]
>>
>> should using --check with --ignore-missing also warn/fail when it
>> verifies no checksum?
>>
>>   $ :|sha1sum |sed s/-/no-such/ |sha1sum --check --ignore-missing; echo $?
>>   0
>
> It's a fair point, but I see the first error as verifying the
> checksum file itself, and so separate functionality.
>
> Related to this is outputting "MISSING" as well as "OK"
> unless --quiet is specified, though I thought the lack
> of "OK" if no files found would be enough indication
> of an issue in the normal usage?

I think a common expected usage of --ignore-missing would be
the case of an SHA1SUM file listing all possibly-verified files for
which it is common to verify only the one or two downloaded files.
In any invocation that ends up ignoring *all* file names, I would
want a loud warning and failure, to be sure that my eyes (and/or
any tool) notice something is wrong.

The absence of an "OK" is far easier to miss than a diagnostic.
At least a few are often expected to be missing, so I see little
value in emitting "MISSING" diagnostics.

Message #25 received at 15604-done <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Jim Meyering <jim <at> meyering.net>
Cc: 15604-done <at> debbugs.gnu.org, Bernhard Voelker <mail <at> bernhard-voelker.de>,
 Luther <lutheroto <at> gmail.com>, Coreutils <coreutils <at> gnu.org>,
 Kamil Paral <kparal <at> redhat.com>
Subject: Re: bug#15604: [coreutils] [PATCH] md5sum: Add option to ignore
 non-existant files
Date: Mon, 23 Nov 2015 17:24:42 +0000

[Message part 1 (text/plain, inline)]

On 23/11/15 16:41, Jim Meyering wrote:
> I think a common expected usage of --ignore-missing would be
> the case of an SHA1SUM file listing all possibly-verified files for
> which it is common to verify only the one or two downloaded files.
> In any invocation that ends up ignoring *all* file names, I would
> want a loud warning and failure, to be sure that my eyes (and/or
> any tool) notice something is wrong.
> 
> The absence of an "OK" is far easier to miss than a diagnostic.
> At least a few are often expected to be missing, so I see little
> value in emitting "MISSING" diagnostics.

Yes I agree. Thinking more, one could have a syntactically correct
checksum file which is adjusted to comment out certain entries, and
currently sha1sum etc. (with or without --ignore-missing)
will error out unless something is verified:

$ echo '#'|sha1sum --check --ignore-missing
sha1sum: standard input: no properly formatted SHA1 checksum lines found

So given that the existing functionality is to ensure something is verified,
then --ignore-missing should be consistent.

I'll push the attached a bit later.

thanks!
Pádraig

[ignore-missing-none.patch (text/x-patch, attachment)]

Message #28 received at 15604-done <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Pádraig Brady <P <at> draigbrady.com>
Cc: 15604-done <at> debbugs.gnu.org, Bernhard Voelker <mail <at> bernhard-voelker.de>,
 Luther <lutheroto <at> gmail.com>, Coreutils <coreutils <at> gnu.org>,
 Kamil Paral <kparal <at> redhat.com>
Subject: Re: bug#15604: [coreutils] [PATCH] md5sum: Add option to ignore
 non-existant files
Date: Mon, 23 Nov 2015 18:53:48 +0100

On Mon, Nov 23, 2015 at 6:24 PM, Pádraig Brady <P <at> draigbrady.com> wrote:
> On 23/11/15 16:41, Jim Meyering wrote:
>> I think a common expected usage of --ignore-missing would be
>> the case of an SHA1SUM file listing all possibly-verified files for
>> which it is common to verify only the one or two downloaded files.
>> In any invocation that ends up ignoring *all* file names, I would
>> want a loud warning and failure, to be sure that my eyes (and/or
>> any tool) notice something is wrong.
>>
>> The absence of an "OK" is far easier to miss than a diagnostic.
>> At least a few are often expected to be missing, so I see little
>> value in emitting "MISSING" diagnostics.
>
> Yes I agree. Thinking more, one could have a syntactically correct
> checksum file which is adjusted to comment out certain entries, and
> currently sha1sum etc. (with or without --ignore-missing)
> will error out unless something is verified:
>
> $ echo '#'|sha1sum --check --ignore-missing
> sha1sum: standard input: no properly formatted SHA1 checksum lines found
>
> So given that the existing functionality is to ensure something is verified,
> then --ignore-missing should be consistent.
>
> I'll push the attached a bit later.

Thank you.
That looks great.

Only suggestions are barely worth mentioning.
When negating, I'm pretty sure the grammar police suggest to use singular:

-            error (0, 0, _("%s: no files were verified"),
+            error (0, 0, _("%s: no file was verified"),

Also, please insert the comma in this log message sentence:

  * src/md5sum.c (digest_check): Update a matched_checksums bool upon
  matched checksum, and fail (loudly unless --status is specified)

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.