GNU bug report logs - #33786
doc: sort: document Debian's version-sort algorithm

Previous Next

To reply to this bug, email your comments to 33786 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: L A Walsh <coreutils <at> tlinx.org>
To: Coreutils <bug-coreutils <at> gnu.org>
Subject: Bug: undocumented feature (algorithm) for version-sort (include on
 manpage)
Date: Mon, 17 Dec 2018 23:11:49 -0800

Recently there was some discussion on inconsistencies in
how version sort worked and some people *basically*, said:

  "it's not our fault, it's Debian's algorithm, you wanna
  change it, convince them."

Um...fine.  Except that it is a Gnu tool, not a Debian tool,
meaning that if one is going to put a Debian sort into a
general purpose tool like "sort", then the algorithm really
needs to be documented.

This means there is no way to verify consistent behavior
from as the utility matures and no way to write an
independent, auditable test case to assure that the sort 
algorithm, operates with consistency from release to release
as well as w/r/t other included sort algorithms.

The request here is for the algorithm used by 'version-sort'
be included in sort's manpage.  This should document
sort's features for reference and use by users who are using
the utility in its native, cmd-line environment.  

Also of importance: that the documentation should be include
with the source and installable with the program executable.

thanks,
linda

Message #14 received at 33786 <at> debbugs.gnu.org (full text, mbox):

From: Assaf Gordon <assafgordon <at> gmail.com>
To: L A Walsh <coreutils <at> tlinx.org>, 33786 <at> debbugs.gnu.org
Subject: Re: bug#33786: Bug: undocumented feature (algorithm) for version-sort
 (include on manpage)
Date: Tue, 18 Dec 2018 01:04:42 -0700

tags  33786 notabug
severity 33786 wishlist
retitle 33786 doc: sort: document Debian's version-sort algorithm
stop

Hello,

On 2018-12-18 12:11 a.m., L A Walsh wrote:
> meaning that if one is going to put a Debian sort into a
> general purpose tool like "sort", then the algorithm really
> needs to be documented.

It is well documented in many places online, e.g.:
https://www.debian.org/doc/debian-policy/ch-controlfields.html#version
https://readme.phys.ethz.ch/documentation/debian_version_numbers/

With a shorter summary available in the coreutils manual here:
https://www.gnu.org/software/coreutils/manual/html_node/Details-about-version-sort.html#Details-about-version-sort

> This means there is no way to verify consistent behavior
> from as the utility matures 

The sort-version.sh test ensure the behavior is consistent from
one release to the next:
https://opengrok.housegordon.com/source/xref/coreutils/tests/misc/sort-version.sh

It also ensures the behavior is compatible with Debian's definition.

> and no way to write an
> independent, auditable test case to assure that the sort algorithm, 
> operates with consistency from release to release
> as well as w/r/t other included sort algorithms.

This message contains example of how to compare results between
coreutils' sort and debian's utilities:
https://lists.gnu.org/archive/html/bug-coreutils/2018-11/msg00017.html

Here's a post about doing the same using python:
https://stackoverflow.com/a/4957741

And in NodeJS:
https://www.npmjs.com/package/deb-version-compare

I'm sure there are many other implementations that
allow easy comparison of one against the other to quickly find
any discrepancies.


As for auditable code, the actual code is here (part of gnulib):
https://opengrok.housegordon.com/source/xref/gnulib/lib/filevercmp.c

And gnulib also includes a unit-test:
https://opengrok.housegordon.com/source/xref/gnulib/tests/test-filevercmp.c

There's no better audit-ability than the source code itself.


> The request here is for the algorithm used by 'version-sort'
> be included in sort's manpage.  This should document
> sort's features for reference and use by users who are using
> the utility in its native, cmd-line environment.

If a coreutils' program implements a known standard,
it's not necessarily beneficial to include implementation details of
the standard, as this is available elsewhere.

For example, the manual for the "base64" program does not include
an explanation of what base64 is. Instead, it links to RFC4648:
https://www.gnu.org/software/coreutils/manual/html_node/base64-
invocation.html#base64-invocation

As your request is for a change in documentation, I'm marking this
as a wish-list item.
As always, concrete patches are welcomed and they go a long way towards
expediting any desired changes - if you have suggestions please do send
a patch.

> Also of importance: that the documentation should be include with the
> source and installable with the program executable.
When someone downloads coreutils' source, they automatically get
the manual (in texinfo format, easily convertible to HTML/PDF).

When they install coreutils (e.g. "make install"), the manual
is also installed (as an "info" file).


regards,
 - assaf

Message #17 received at 33786 <at> debbugs.gnu.org (full text, mbox):

From: L A Walsh <coreutils <at> tlinx.org>
To: Assaf Gordon <assafgordon <at> gmail.com>
Cc: 33786 <at> debbugs.gnu.org
Subject: Re: bug#33786: Bug: undocumented feature (algorithm) for version-sort
 (include on manpage)
Date: Tue, 18 Dec 2018 00:06:22 -0800

So undocumented features are considered wishlist items
in Gnu?   

On 12/18/2018 12:04 AM, Assaf Gordon wrote:
> tags  33786 notabug
> severity 33786 wishlist
> retitle 33786 doc: sort: document Debian's version-sort algorithm
> stop
> 
> Hello,
> 
> On 2018-12-18 12:11 a.m., L A Walsh wrote:
>> meaning that if one is going to put a Debian sort into a
>> general purpose tool like "sort", then the algorithm really
>> needs to be documented.
> 
> It is well documented in many places online, e.g.:
> https://www.debian.org/doc/debian-policy/ch-controlfields.html#version
> https://readme.phys.ethz.ch/documentation/debian_version_numbers/
> 
> With a shorter summary available in the coreutils manual here:
> https://www.gnu.org/software/coreutils/manual/html_node/Details-about-version-sort.html#Details-about-version-sort
> 
>> This means there is no way to verify consistent behavior
>> from as the utility matures 
> 
> The sort-version.sh test ensure the behavior is consistent from
> one release to the next:
> https://opengrok.housegordon.com/source/xref/coreutils/tests/misc/sort-version.sh
> 
> It also ensures the behavior is compatible with Debian's definition.
> 
>> and no way to write an
>> independent, auditable test case to assure that the sort algorithm, 
>> operates with consistency from release to release
>> as well as w/r/t other included sort algorithms.
> 
> This message contains example of how to compare results between
> coreutils' sort and debian's utilities:
> https://lists.gnu.org/archive/html/bug-coreutils/2018-11/msg00017.html
> 
> Here's a post about doing the same using python:
> https://stackoverflow.com/a/4957741
> 
> And in NodeJS:
> https://www.npmjs.com/package/deb-version-compare
> 
> I'm sure there are many other implementations that
> allow easy comparison of one against the other to quickly find
> any discrepancies.
> 
> 
> As for auditable code, the actual code is here (part of gnulib):
> https://opengrok.housegordon.com/source/xref/gnulib/lib/filevercmp.c
> 
> And gnulib also includes a unit-test:
> https://opengrok.housegordon.com/source/xref/gnulib/tests/test-filevercmp.c
> 
> There's no better audit-ability than the source code itself.
> 
> 
>> The request here is for the algorithm used by 'version-sort'
>> be included in sort's manpage.  This should document
>> sort's features for reference and use by users who are using
>> the utility in its native, cmd-line environment.
> 
> If a coreutils' program implements a known standard,
> it's not necessarily beneficial to include implementation details of
> the standard, as this is available elsewhere.
> 
> For example, the manual for the "base64" program does not include
> an explanation of what base64 is. Instead, it links to RFC4648:
> https://www.gnu.org/software/coreutils/manual/html_node/base64-
> invocation.html#base64-invocation
> 
> As your request is for a change in documentation, I'm marking this
> as a wish-list item.
> As always, concrete patches are welcomed and they go a long way towards
> expediting any desired changes - if you have suggestions please do send
> a patch.
> 
>> Also of importance: that the documentation should be include with the
>> source and installable with the program executable.
> When someone downloads coreutils' source, they automatically get
> the manual (in texinfo format, easily convertible to HTML/PDF).
> 
> When they install coreutils (e.g. "make install"), the manual
> is also installed (as an "info" file).
> 
> 
> regards,
>   - assaf

Message #20 received at submit <at> debbugs.gnu.org (full text, mbox):

From: L A Walsh <coreutils <at> tlinx.org>
To: Assaf Gordon <assafgordon <at> gmail.com>
Cc: Coreutils <bug-coreutils <at> gnu.org>
Subject: Re: bug#33786: Bug: undocumented feature (algorithm) for version-sort
 (include on manpage)
Date: Tue, 18 Dec 2018 00:13:34 -0800

I wouldn't consider debian to be a standards organization.

Perhaps they should get their version sort adopted by POSIX?

But having algorithms that read:

use ascii sort for this field if after a character in this list [].
but use numeric sort for this field if after this list [].
but use an indeterminant sort if the field has unicode characters...

would hardly seem likely if it was a publish standard.

In the test cases I used, subsequent fields' sort methods
were determined by previous fields with a non-obvious
behavior. 

That it is well documented on the net 'somewhere', is 
part of the problem -- on a console, there was no
web browser nor internet access.

Message #23 received at 33786 <at> debbugs.gnu.org (full text, mbox):

From: Assaf Gordon <assafgordon <at> gmail.com>
To: L A Walsh <coreutils <at> tlinx.org>
Cc: 33786 <at> debbugs.gnu.org
Subject: Re: bug#33786: Bug: undocumented feature (algorithm) for version-sort
 (include on manpage)
Date: Tue, 18 Dec 2018 01:23:07 -0700

Hello,

On 2018-12-18 1:06 a.m., L A Walsh wrote:
> So undocumented features are considered wishlist items
> in Gnu?

In your message you wrote:

On 2018-12-18 12:11 a.m., L A Walsh wrote:
> The request here is for the algorithm used by 'version-sort' be
> included in sort's manpage.

Thus it is a request for a future improvement - not a report about
a bug in an existing program or a bug (=incorrect information) in
the current manual.

In the parlance of the DebBugs system, it is a "wishlist" item
(as opposed to a bug), see here:
https://debbugs.gnu.org/Developer.html#severities

This item remains open ( https://bugs.gnu.org/33786 ).
It is not closed as "notabug", and not rejected as "wontfix".
Thus, it will stay active  until someone takes the time to address the
issue of documenting the algorithm, or until it is decided that
there's no interest in such change.

---

Regarding "undocumented feature", as I wrote in the previous message,
the algorithm is well documented in other place online (and indeed,
as you pointed, not documented in exact details in the coreutils
manual).

I think this is quite different from the typical definition of an
"undocumented feature" (e.g. a hidden feature that is not intended
to be used by most end-users).

---

Then again,
If you or anyone else have concrete suggestions to improve the manual,
please do send a patch.

regards,
 - assaf

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.