GNU bug report logs - #9987
RFE: 'groups' command ADD command switches "-0", and "-1"

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 9987 in the body.
You can then email your comments to 9987 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Linda Walsh <coreutils <at> tlinx.org>
To: bug-coreutils <at> gnu.org
Subject: RFE: 'groups' command ADD command switches "-0", and "-1"
Date: Mon, 07 Nov 2011 14:27:50 -0800

I'd like to request an RFE for the groups command to add 2 switches:

-1 - print groups  in 1 column (cf. ls -1)
-0 - print groups followed by null (cf. find et al)

reasons: have groups with spaces in them.
             provide ultimate safety/futureproof with -0

??
reasonable?

Message #8 received at 9987 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Linda Walsh <coreutils <at> tlinx.org>
Cc: 9987 <at> debbugs.gnu.org
Subject: Re: bug#9987: RFE: 'groups' command ADD command switches "-0", and
	"-1"
Date: Tue, 08 Nov 2011 00:26:34 +0000

On 11/07/2011 10:27 PM, Linda Walsh wrote:
> 
> 
> 
> I'd like to request an RFE for the groups command to add 2 switches:
> 
> -1 - print groups  in 1 column (cf. ls -1)
> -0 - print groups followed by null (cf. find et al)
> 
> reasons: have groups with spaces in them.
>              provide ultimate safety/futureproof with -0
> 
> ??
> reasonable?

Hmm. I suppose by extension that `id -Gn` should accept -1 too.
But do you really want to support group names with spaces?
`groupadd` for example won't allow this.
I suppose integration with LDAP etc. might get arbitrary group names.

Note POSIX is quite specific about the output format for `id`:

"−G Output all different group IDs (effective, real, and supplementary) only, using the
format "%u\n". If there is more than one distinct group affiliation, output each
such affiliation, using the format " %u", before the <newline> is output."

That suggests to me that \n is used to distinguish the three types of IDs.
So -1 would have to override that functionality.

cheers,
Pádraig.

Message #11 received at 9987 <at> debbugs.gnu.org (full text, mbox):

From: "Linda A. Walsh" <lkml <at> tlinx.org>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 9987 <at> debbugs.gnu.org
Subject: Re: bug#9987: RFE: 'groups' command ADD command switches "-0",
	and "-1"
Date: Mon, 07 Nov 2011 17:38:38 -0800

Pádraig Brady wrote:

> On 11/07/2011 10:27 PM, Linda Walsh wrote:
>>
>>
>> I'd like to request an RFE for the groups command to add 2 switches:
>>
>> -1 - print groups  in 1 column (cf. ls -1)
>> -0 - print groups followed by null (cf. find et al)
>>
>> reasons: have groups with spaces in them.
>>              provide ultimate safety/futureproof with -0
>>
>> ??
>> reasonable?
>
> Hmm. I suppose by extension that `id -Gn` should accept -1 too.
> But do you really want to support group names with spaces?
> `groupadd` for example won't allow this.
> I suppose integration with LDAP etc. might get arbitrary group names.

----
	I already have groups and usernames with spaces in them.

	Just try merging/supporting a Windows environment via linux.

I EVEN have usernames with "\" in them!...

Was required for 'ssh' to work...since when I'm validated as 'user' via 
samba, when I 'ssh' in from a domain workstation, it comes in as user 
"Domain\user".

I just added an extra PW entry for Domain\me, same as "me", and it works...

Fortunately most core utils are mostly agnostic towards these things... 
it's when you get to talking to people who who tell you that spaces and 
backslashes don't belong there cuz God didn't intend it, that you get into
'religious' discussions'....

Anything other a a null (or colon) should be fair game for user/groupnames 
-- I'd feel uncomfortable about control chars, but hey. no reason 
*technically*, why they should be disallowed (policy is separate from 
technical implementation! ;-)).

NT does Unix 1 better -- they alway use a count for strings.  So even 
'nulls' are ok... though windows, like linux uses null-terminated strings.

This leads to interesting hacks in windows where NT-progs can create 
reg-entries and files that Windows progs can't touch/see due to embedded 
nulls.

Might lead to some interesting file portability probs for NT files created 
with an NT inteface, say on an NTFS file system (dunno about FAT), that 
linux couldn't see -- might even be a way for NT to hide stuff from 
linux...especially since linux supports NTFS and FAT...probably not
an issue in FAT.  I'm unsure about NTFS though as it uses the same
naming rules as the registry, I don't see why it wouldn't have the same
"gotcha's".  I do know it's a prob in the registry...various vendors like 
to hide registration data with embedded nulls in them so they can't be 
touched by normal user processes (ms-sysinternals/regdelnull tool will 
wipe them...but not make them readable...would rather it change the nulls 
to spaces or such...)...


>
> Note POSIX is quite specific about the output format for `id`:
---
	Which posix?  there are about 3-4 different versions.

They are NOT compatible.  I.e. a shell script written against the 1988 
(1990?) spec won't necessarily work against the 2008 spec....etc.


	So you can put alot of stock in POSIX, but you better specify which POSIX 
you are talking about ...  as for me, as soon as POSIX became
incompatible with POSIX, i realized, that they'd "lost the way".... ;-)


>
> "−G Output all different group IDs (effective, real, and supplementary) 
only, using the
> format "%u\n". If there is more than one distinct group affiliation, 
output each
> such affiliation, using the format " %u", before the <newline> is output."

Um...   I don't think the same problem would exist in put out 'uid'/rid' 
which are 32-bit unsigned integers, vs. 'UTF-8 encoded strings' for user 
names......   Two different problem spaces...

I mean you *could* add those options for numeric output, but, personally, 
I don't think that would be a logical step.

Message #14 received at 9987 <at> debbugs.gnu.org (full text, mbox):

From: "Linda A. Walsh" <lkml <at> tlinx.org>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 9987 <at> debbugs.gnu.org
Subject: Re: bug#9987: RFE: 'groups' command ADD command switches "-0",
	and "-1"
Date: Mon, 07 Nov 2011 18:04:51 -0800

Pádraig Brady wrote:

> Hmm. I suppose by extension that `id -Gn` should accept -1 too.
> But do you really want to support group names with spaces?
> `groupadd` for example won't allow this.
> I suppose integration with LDAP etc. might get arbitrary group names.

... i.e. some don't have names (thanks samba/winbind!)...
Some map to BUILTIN\xxxx

Since I'm logged into the Domain Controller, I wouldn't expect to see the 
Domainname,
FWIW, when I type in 'id', I get:
> id
uid=5001(lindaw) gid=201(lwgroup) 
groups=201(lgroup),0(root),5(tty),6(disk),8(mail),
10(wheel),12(games),14(uucp),16(dialout),17(audio),18(SYSGROUP),
20(cdrom),30(wwwrun),31(squid),42(trusted),44(named),76(ldap),100(backup),132(spamd),
133(video),200(unix_users),202(home),203(private),204(powerd),212(mailadmin),213(devel),
215(lighttpd),225(yard),231(Juno),232(scan),237(cronusers),238(atusers),260(torrent),
512(Domain Admins),513(Domain Users),517(Cert Publishers),
518(Schema Admins),519(Enterprise Admins),544(Administrators),
545(Users),547(Power Users),548(Account Operators),
551(Backup Operators),555(Remote Desktop Users),
558(Performance Monitor Users),
5001(lindaw),10001,10002,10003,10004,10005,10006,10007,
10008,10010,10011,10012,10013,10014,10015,10016,10017,10018,
10019,10020,10021,10022,10023,10024,10025,10026,10027,10028,
10029,10030,10031,10032,10046,10051(BUILTIN\administrators),
10052(BUILTIN\users),10060,20001(media),
50001(samba),50002(Trusted Local Net Users),
50003(CPAN)

----
But on my Windows 7 workstation, under cygwin, I type in 'id' and I see:

uid=90026(Bliss\lindaw) gid=71008(Bliss\lwgroup) 
groups=71008(Bliss\lwgroup),0(root),544(Administrators),
555(Remote Desktop Users),545(Users),1005(lwgroup),
11023(Bliss\torrent),11017(Bliss\media),
60002(Bliss\Trusted Local Net Users),
10512(Bliss\Domain Admins),11039(Bliss\Unix Group root)

(none of the local groups are mapped over, but some of the the groups are 
'domain group' and get mapped over...)...

See and there's that 'Bliss\lindaw', on cygwin, that I had to deal with on 
linux...

Of course, another problem Samba causes is it mangles the case names, -- 
windows partly cares and linux/unix REALLY cares....   I've tried to get
them to change that, but they pretty much threw up there hands and said 
'too big a bug to fix...break it down into small pieces for us'...
(since it involves user names, groupnames, domain names, hostnames, etc...

Windows is case preserving but ignoring, vs. samba is case mangling (upper 
or lower depending on what it 'thinks' it is, which is great, since user 
and host names get mangled in opposite directions, but hostnames have to 
be registered as usernames ...Of course you have "nimnils" like me, who
are silly enough to use 'Upper and Lower case...and boy ... it *USED* to
work for years...but that was before ...about 6 months ago...I think they 
either changed something in samba, in windows or both...

Have seen more than one notation on the internet about how MS has done 
things to break samba compatibility over the years...Now they are both
allowing mixed case, and in some cases treating it differently...

My workstation is listed as belonging to workgroup 'BLISS', but domain
'Bliss' (my domain name took it's name from the initial XP background 
(bliss.jpg/bliss.scr)...green grass blue sky...)...It was meant as 
'irony..'Bliss...& windows...'...oi!


--------------------------


Using a somewhat reverse mode of attack,in the Music-labeling space, MS 
media player is refusing to play any songs labeled in UTF-8...has to be 
UTF-16 (which is also illegal, but gets by because it's too hard to tell 
from USC-2) which is what is allowed)....  So there they are shutting out
products that don't conform to their standard (of course interestingly the 
id3v2.org folks are well padded with MS-centric standards and lingo...
Gee..I wonder who sponsors them...

Message #19 received at 9987 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Ondrej Oprala <ooprala <at> redhat.com>
Cc: 9987 <at> debbugs.gnu.org
Subject: Re: [PATCH] id, groups: optionally delimit output with null or newline
Date: Tue, 28 May 2013 11:40:36 +0100

unarchive 9987
stop

On 05/28/2013 10:10 AM, Ondrej Oprala wrote:
> Hi,
> I'd like to propose a patch based on http://lists.gnu.org/archive/html/bug-coreutils/2011-11/msg00024.html .
> The -{0,1} option is only enabled for some option combinations (as discussed in above link)
> although I have some doubts about the usefulness of groups -0|-1 without any other arguments;
> on the other hand, changing the delims for the "USERNAME_1: group_1...group_n" output is IMHO very weird.
> However, maybe we could only print the "USERNAME: " part if more than one username is specified as an argument.
> Or does "id USERNAME -Gn" suffice in this case? What do you think?

Thanks for following this up.

> diff --git a/src/groups.c b/src/groups.c
> index 53332d5..8f5a714 100644

>    if (optind == argc)
>      {

>      }
>    else
>      {
> +      if (nullterm || one_per_line)
> +        error (EXIT_FAILURE, 0, _("delimiter modififers cannot be "
> +                                     "combined with other arguments"));

So what you're saying here is that when groups(1) goes into
its alternate display mode when multiple users are specified,
then discount specified delimiters in this case.
I'd concur with that, since this mode would mainly be
for human rather than programmatic consumption.

So all we have to consider is `groups [$USER]` and 'id -Gn`.
I notice that groups goes into "$user: $group1...$groupn" mode
even if a single user is specified. I notice solaris or FreeBSD
do not do that. As part of this as a separate patch, I'd be
inclined to behave like solaris/BSD here so that programmatic
access to groups for all users would be improved.

As for which of -01z to use.
Do we really need -1?
That could be trivially achieved with tr '\0' '\n'
Given that this is to aid programmatic access to group names,
I'd be inclined to not support -1

As for -0 vs -z:
-0, --null: du, env, printenv
-z, --zero: basename, dirname, join, sort, uniq, readlink, realpath, shuf

So for consistency I'd use -z, --zero (adding the long option too).

thanks,
Pádraig.

Message #22 received at 9987 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Bernhard Voelker <mail <at> bernhard-voelker.de>
Cc: 9987 <at> debbugs.gnu.org
Subject: Re: [PATCH] groups,id: add -0, --null option
Date: Mon, 02 Sep 2013 15:03:20 +0100

unarchive 9987
stop

On 09/01/2013 10:35 PM, Bernhard Voelker wrote:
> The attached patch implements the -0,--null option for groups(1)
> and id(1) as discussed in
> 
>   http://bugs.gnu.org/9987

bug CC'd

> 
> and
> 
>   http://lists.gnu.org/archive/html/bug-coreutils/2011-11/msg00024.html
> 
> and as mentioned on Padraig's TODO list:
> 
>   http://www.pixelbeat.org/patches/coreutils/inbox_apr_2013.html
>   TODO --> "group -0 to support group names with spaces etc."

I'd not updated the TODO, but the details in 9987 above
suggested that -z, --zero would be better since most tools use that.

Also mentioned there was only honoring -z when the groups for
a single user were being output. I.E. no double NUL stuff.
But I'm not against the double NUL approach really.

It would be good I think to have a separate patch though
that aligned with other systems and didn't output the
"user:" at the start of the line when only a single user is specified.

thanks,
Pádraig.

Message #25 received at 9987 <at> debbugs.gnu.org (full text, mbox):

From: Bernhard Voelker <mail <at> bernhard-voelker.de>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 9987 <at> debbugs.gnu.org
Subject: Re: bug#9987: [PATCH] groups,id: add -0, --null option
Date: Tue, 03 Sep 2013 08:35:28 +0200

On 09/02/2013 04:03 PM, Pádraig Brady wrote:
> I'd not updated the TODO, but the details in 9987 above
> suggested that -z, --zero would be better since most tools use that.

I'd personally prefer -0 because burning the -z short option may
be problematic for future extensions while the only use of
the -0 option in any program is "separate output by NULs".
But I'm game if you prefer -z.

> Also mentioned there was only honoring -z when the groups for
> a single user were being output. I.E. no double NUL stuff.
> But I'm not against the double NUL approach really.

thanks, then I'll keep it.

> It would be good I think to have a separate patch though
> that aligned with other systems and didn't output the
> "user:" at the start of the line when only a single user is specified.

Interestingly, groups(1) doesn't seem to be specified by the
OpenGroup documents, only id(1):
  http://pubs.opengroup.org/onlinepubs/9699919799/utilities/id.html
Is there any other resource?

But I agree, outputting the "user:" prefix for only one argument
seems to be silly.

Thanks!

Have a nice day,
Berny

Message #28 received at 9987 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Bernhard Voelker <mail <at> bernhard-voelker.de>
Cc: 9987 <at> debbugs.gnu.org
Subject: Re: bug#9987: [PATCH] groups,id: add -0, --null option
Date: Mon, 16 Sep 2013 01:18:28 +0100

On 09/03/2013 07:35 AM, Bernhard Voelker wrote:
> On 09/02/2013 04:03 PM, Pádraig Brady wrote:
>> I'd not updated the TODO, but the details in 9987 above
>> suggested that -z, --zero would be better since most tools use that.
> 
> I'd personally prefer -0 because burning the -z short option may
> be problematic for future extensions while the only use of
> the -0 option in any program is "separate output by NULs".
> But I'm game if you prefer -z.

Yes please use -z, --zero for most consistency.
Note also the -O0 display issue and --nul being more accurate than --null

>> Also mentioned there was only honoring -z when the groups for
>> a single user were being output. I.E. no double NUL stuff.
>> But I'm not against the double NUL approach really.
> 
> thanks, then I'll keep it.
> 
>> It would be good I think to have a separate patch though
>> that aligned with other systems and didn't output the
>> "user:" at the start of the line when only a single user is specified.
> 
> Interestingly, groups(1) doesn't seem to be specified by the
> OpenGroup documents, only id(1):
>   http://pubs.opengroup.org/onlinepubs/9699919799/utilities/id.html
> Is there any other resource?
> 
> But I agree, outputting the "user:" prefix for only one argument
> seems to be silly.

If you could handle that as a separate patch it would be great.

thanks,
Pádraig.

Message #31 received at 9987 <at> debbugs.gnu.org (full text, mbox):

From: Bernhard Voelker <mail <at> bernhard-voelker.de>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 9987 <at> debbugs.gnu.org
Subject: Re: bug#9987: [PATCH] groups,id: add -0, --null option
Date: Mon, 16 Sep 2013 07:40:27 +0200

On 09/16/2013 02:18 AM, Pádraig Brady wrote:
> Yes please use -z, --zero for most consistency.

Ok, fine, thanks.


>>> It would be good I think to have a separate patch though
>>> that aligned with other systems and didn't output the
>>> "user:" at the start of the line when only a single user is specified.

> If you could handle that as a separate patch it would be great.

I'll do, no problem.

Thanks & have a nice day,
Berny

Message #34 received at 9987 <at> debbugs.gnu.org (full text, mbox):

From: Bernhard Voelker <mail <at> bernhard-voelker.de>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 9987 <at> debbugs.gnu.org
Subject: Re: bug#9987: [PATCH] groups,id: add -0, --null option
Date: Tue, 17 Sep 2013 00:21:35 +0200

[Message part 1 (text/plain, inline)]

On 09/16/2013 07:40 AM, Bernhard Voelker wrote:
> On 09/16/2013 02:18 AM, Pádraig Brady wrote:
>> Yes please use -z, --zero for most consistency.
> 
> Ok, fine, thanks.

Here comes the new patch with {groups,id} -z,--zero.

Have a nice day,
Berny

[groups-id-zero.patch (text/x-patch, attachment)]

Message #37 received at 9987 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Bernhard Voelker <mail <at> bernhard-voelker.de>
Cc: 9987 <at> debbugs.gnu.org
Subject: Re: bug#9987: [PATCH] groups,id: add -0, --null option
Date: Tue, 17 Sep 2013 00:19:44 +0100

On 09/16/2013 11:21 PM, Bernhard Voelker wrote:
> On 09/16/2013 07:40 AM, Bernhard Voelker wrote:
>> On 09/16/2013 02:18 AM, Pádraig Brady wrote:
>>> Yes please use -z, --zero for most consistency.
>>
>> Ok, fine, thanks.
> 
> Here comes the new patch with {groups,id} -z,--zero.
> 
> Have a nice day,
> Berny
> 

Very nice.
I can't find any issues with that at all.

thanks!
Pádraig.

Message #40 received at 9987 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Bernhard Voelker <mail <at> bernhard-voelker.de>
Cc: 9987 <at> debbugs.gnu.org
Subject: Re: bug#9987: [PATCH] groups,id: add -0, --null option
Date: Tue, 17 Sep 2013 00:32:40 +0100

On 09/17/2013 12:19 AM, Pádraig Brady wrote:
> On 09/16/2013 11:21 PM, Bernhard Voelker wrote:
>> On 09/16/2013 07:40 AM, Bernhard Voelker wrote:
>>> On 09/16/2013 02:18 AM, Pádraig Brady wrote:
>>>> Yes please use -z, --zero for most consistency.
>>>
>>> Ok, fine, thanks.
>>
>> Here comes the new patch with {groups,id} -z,--zero.
>>
>> Have a nice day,
>> Berny
>>
> 
> Very nice.
> I can't find any issues with that at all.

Actually omitting the trailing \0 could be
inconsistent and problematic?


uniq and basename for example always output the trailing \0

$ basename -az 1/1 2/2 | od -Ax -tx1z -v
000000 31 00 32 00                                      >1.2.<
000004

$ printf "1\0002" | uniq -z | od -Ax -tx1z -v
000000 31 00 32 00                                      >1.2.<
000004


Also handling only separated output in shell can be awkward:

# This is OK:
$ groups | tr ' ' '\n' |  while read group; do echo $group; done | wc -l
4

# This drops the last item
$ groups -z | tr '\0' '\n' | while read group; do echo $group; done | wc -l
3

cheers,
Pádraig.

Message #43 received at 9987 <at> debbugs.gnu.org (full text, mbox):

From: Bernhard Voelker <mail <at> bernhard-voelker.de>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 9987 <at> debbugs.gnu.org
Subject: Re: bug#9987: [PATCH] groups,id: add -0, --null option
Date: Tue, 17 Sep 2013 19:23:19 +0200

On 09/17/2013 01:32 AM, Pádraig Brady wrote:
> Actually omitting the trailing \0 could be
> inconsistent and problematic?
> 
> 
> uniq and basename for example always output the trailing \0
> 
> $ basename -az 1/1 2/2 | od -Ax -tx1z -v
> 000000 31 00 32 00                                      >1.2.<
> 000004
> 
> $ printf "1\0002" | uniq -z | od -Ax -tx1z -v
> 000000 31 00 32 00                                      >1.2.<
> 000004
> 
> 
> Also handling only separated output in shell can be awkward:
> 
> # This is OK:
> $ groups | tr ' ' '\n' |  while read group; do echo $group; done | wc -l
> 4
> 
> # This drops the last item
> $ groups -z | tr '\0' '\n' | while read group; do echo $group; done | wc -l
> 3

Good point.
The changes in id.c and in groups.c are quite easy and make the changes
smaller there.  The bigger work is in the test.  To avoid double work,
I'll therefore prepare the other change first, i.e. removing the
"USERNAME : " prefix in the output of groups in the case
where only one argument was passed).

Thanks & have a nice day,
Berny

Message #46 received at 9987 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Bernhard Voelker <mail <at> bernhard-voelker.de>
Cc: 9987 <at> debbugs.gnu.org
Subject: Re: bug#9987: [PATCH] groups,id: add -0, --null option
Date: Wed, 18 Sep 2013 10:43:36 +0100

tl;dr

The is about being able to get a simple list of a
particular user's groups by doing `groups user`.
However it's probably best to not change the existing
output format of `groups` for a single argument
as we can just use `id` for this functionality.

On 09/16/2013 01:18 AM, Pádraig Brady wrote:
> On 09/03/2013 07:35 AM, Bernhard Voelker wrote:
>> On 09/02/2013 04:03 PM, Pádraig Brady wrote:

>>> It would be good I think to have a separate patch though
>>> that aligned with other systems and didn't output the
>>> "user:" at the start of the line when only a single user is specified.
>>
>> Interestingly, groups(1) doesn't seem to be specified by the
>> OpenGroup documents, only id(1):
>>   http://pubs.opengroup.org/onlinepubs/9699919799/utilities/id.html
>> Is there any other resource?
>>
>> But I agree, outputting the "user:" prefix for only one argument
>> seems to be silly.
> 
> If you could handle that as a separate patch it would be great.

On 09/17/2013 10:12 PM, Bernhard Voelker wrote:
> Hi Padraig,
>
> you wrote that it would be good to do this to "align with other
> systems". What systems do you refer to here?

Solaris and BSD at least. GNU is the outlier here.

>
> After thinking about it, I don't have a good feeling about it:
> a) such a change would possibly break existing scripts.
>    Okay, groups has never been intended to be used primarily
>    in scripts, but hey, we're planning to add --zero for that.

That's a fair point, but even such scripts might be immune.
If I wanted to process a list of user groups I would:

for user in 'me' 'you'; do
  for group in $(groups $user | sed 's/.*: //'); do
     echo processing $user:$group
  done
done

The sed manipulation still works with the proposed change.

> b) the new behavior would not be predictable anymore
>    in cases like
>      echo "$listofusers" | xargs groups
>    because xargs could exec a new groups process with just
>    one last argument left.

This is a fair point and would break scripts that did:

groups 'me' 'maybe_you' |
while read user _sep groups; do
  for group in $groups; do
    echo processing $user:$group
  done
done

Drats so changing output format based on the number of inputs is quite problematic.

We had the same tradeoff when adjusting basename to support multiple arguments.
On FreeBSD you must pass -a to get basename to process 2 args:
bsd$ basename 1 2
1
bsd$ basename 1 2 3
1
2
3
bsd$ basename -a 1 2
1
2

So we didn't mimic that behavior for the same reason,
and required the -a option to process all args:

GNU$ basename 1 2 3
basename: extra operand ‘3’
GNU$ basename -a 1 2 3
1
2
3

So how would we best select between the two output formats?
I wonder could we first look for the $USER env variable,
and if not empty, use getpwnam($USER) rather than getuid() etc.
So then you could select the simple output format for a specific user like:

  USER=whatever groups

I'm worried of some cases though where USER might not be accurate,
and it's probably not portable (USERNAME and LOGNAME used elsewhere).

So we're probably best use an option if we're going to change this at all.
If we were to do that --user would probably be best.

However we already have this functionality from `id -Gn` so it's not
worth change `groups` to do this.

I'll send a doc patch later for the groups command to summarize to
use `id -Gn` for cases like this, and in general since it's more
portable and POSIX standardized.

thanks!
Pádraig.

Message #49 received at 9987 <at> debbugs.gnu.org (full text, mbox):

From: Bernhard Voelker <mail <at> bernhard-voelker.de>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 9987 <at> debbugs.gnu.org
Subject: Re: bug#9987: [PATCH] groups,id: add -0, --null option
Date: Wed, 18 Sep 2013 20:46:25 +0200

On 09/18/2013 11:43 AM, Pádraig Brady wrote:
> However we already have this functionality from `id -Gn` so it's not
> worth change `groups` to do this.

Hi Padraig,

thanks for confirming this - using groups(1) is already not enough
portable among platforms.

Regarding the main discussion point, i.e. adding --zero option:
I see the following alternatives:
a) add the -z option only to id.c leaving groups(1) untouched.
b) add the -z option to id.c and groups.c, but don't allow it
   only if no USERNAME argument is given.
c) add the -z option to id.c and groups.c for all cases (including
   the unusual "double NUL" separator between the output for
   consecutive user arguments).

To reduce further incompatibilities, I'd vote for b), i.e. adding
it to groups(1) only for the case where it has to compatible with
"id -Gn".

Other opinions?

Have a nice day,
Berny

Message #52 received at 9987 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Bernhard Voelker <mail <at> bernhard-voelker.de>
Cc: 9987 <at> debbugs.gnu.org
Subject: Re: bug#9987: [PATCH] groups,id: add -0, --null option
Date: Wed, 18 Sep 2013 20:56:05 +0100

On 09/18/2013 07:46 PM, Bernhard Voelker wrote:
> On 09/18/2013 11:43 AM, Pádraig Brady wrote:
>> However we already have this functionality from `id -Gn` so it's not
>> worth change `groups` to do this.
> 
> Hi Padraig,
> 
> thanks for confirming this - using groups(1) is already not enough
> portable among platforms.
> 
> Regarding the main discussion point, i.e. adding --zero option:
> I see the following alternatives:
> a) add the -z option only to id.c leaving groups(1) untouched.
> b) add the -z option to id.c and groups.c, but don't allow it
>    only if no USERNAME argument is given.

s/don't//

> c) add the -z option to id.c and groups.c for all cases (including
>    the unusual "double NUL" separator between the output for
>    consecutive user arguments).
> 
> To reduce further incompatibilities, I'd vote for b), i.e. adding
> it to groups(1) only for the case where it has to compatible with
> "id -Gn".
> 
> Other opinions?

I'd vote for a) actually.
That gives more incentive for people to use the standardised id(1).
groups(1) has no options at present, so I'd leave it as is.

thanks,
Pádraig

Message #55 received at 9987 <at> debbugs.gnu.org (full text, mbox):

From: Bernhard Voelker <mail <at> bernhard-voelker.de>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 9987 <at> debbugs.gnu.org
Subject: Re: bug#9987: [PATCH] groups,id: add -0, --null option
Date: Thu, 19 Sep 2013 01:24:08 +0200

[Message part 1 (text/plain, inline)]

On 09/18/2013 09:56 PM, Pádraig Brady wrote:
> On 09/18/2013 07:46 PM, Bernhard Voelker wrote:
>> a) add the -z option only to id.c leaving groups(1) untouched.

> I'd vote for a) actually.

Great, that makes the change much smaller. ;-)

The patch is attached.

Thanks & have a nice day,
Berny

[id-zero-option.patch (text/x-patch, attachment)]

Message #58 received at 9987 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Bernhard Voelker <mail <at> bernhard-voelker.de>
Cc: 9987 <at> debbugs.gnu.org
Subject: Re: bug#9987: [PATCH] groups,id: add -0, --null option
Date: Thu, 19 Sep 2013 02:31:18 +0100

On 09/19/2013 12:24 AM, Bernhard Voelker wrote:
> On 09/18/2013 09:56 PM, Pádraig Brady wrote:
>> > On 09/18/2013 07:46 PM, Bernhard Voelker wrote:
>>> >> a) add the -z option only to id.c leaving groups(1) untouched.
>> > I'd vote for a) actually.
> Great, that makes the change much smaller. ;-)
> 
> The patch is attached.

> diff --git a/NEWS b/NEWS
> index d26722d..5891def 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -44,6 +44,9 @@ GNU coreutils NEWS                                    -*- outline -*-
>    du accepts a new option: --inodes to show the number of inodes instead
>    of the blocks used.
>  
> +  id accepts a new option: --zero (-z) to separate the output entries by
> +  a NUL instead of a white space character.

s/separate/delimit/

>    id and ls with -Z report the SMACK security context where available.
>    mkdir, mkfifo and mknod with -Z set the SMACK context where available.
>  
> diff --git a/doc/coreutils.texi b/doc/coreutils.texi
> index 21216b4..327254b 100644
> --- a/doc/coreutils.texi
> +++ b/doc/coreutils.texi
> @@ -14516,9 +14516,20 @@ Print only the security context of the current user.
>  If SELinux is disabled then print a warning and
>  set the exit status to 1.
>  
> -@end table
> +@item -z
> +@itemx --zero
> +@opindex -z
> +@opindex --zero
> +Separate output items with NUL characters.

s/Separate/Delimit/

> +This option is not permitted when using the default format.
>  
> -@exitstatus
> +Example:
> +@example
> +$ id -Gn --zero
> +users <NUL> devs

users <NUL> devs <NUL>

> diff --git a/tests/misc/id-zero.sh b/tests/misc/id-zero.sh

> +id root || fail=1

Is 'root' a guaranteed name?
misc/chroot-credentials.sh since v7.4-16-gc45c51f assumes so.
I'm still not convinced (and will look at addressing that separately).
Do we even need to treat root specially. Can we just drop explicit root entry?

> +# Create a nice list of users (ignoring errors) ...
> +getent passwd | head -n 100 | cut -d: -f1 > users

test -s users || skip_ 'Failed to read the passwd database with getent'
Also why so many (100). The test is a bit slow and you would get
the same coverage from a couple of entries?

> +# ... and ensure it contains at least 'root', "$u", and an
> +# empty line for the tests without a user argument below.
> +printf "%s\n" root "$u" '' >> users || framework_failure_
> +
> +# id(1) should refuse --zero in default format.
> +id --zero > out 2>err && fail=1
> +compare /dev/null out || fail=1

> +grep 'option --zero not permitted in default format' err || fail=1

Better to:
echo 'option ...' > exp-err
compare exp-err err || fail=1

> +
> +# Exercise "id -z" with various options.
> +printf "\n" > exp || framework_failure_

Minor point, but it's better to use '' quotes when not interpolating,
here and throughout the test.

> +cp /dev/null  out || framework_failure_

:> out

thanks!
Pádraig.

Message #61 received at 9987 <at> debbugs.gnu.org (full text, mbox):

From: Bernhard Voelker <mail <at> bernhard-voelker.de>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 9987 <at> debbugs.gnu.org
Subject: Re: bug#9987: [PATCH] groups,id: add -0, --null option
Date: Thu, 19 Sep 2013 09:56:48 +0200 (CEST)

> On September 19, 2013 at 3:31 AM Pádraig Brady <P <at> draigBrady.com> wrote:
> 
> On 09/19/2013 12:24 AM, Bernhard Voelker wrote:
> > diff --git a/NEWS b/NEWS

> > +  id accepts a new option: --zero (-z) to separate the output entries by
> > +  a NUL instead of a white space character.
> 
> s/separate/delimit/

done.

> > diff --git a/doc/coreutils.texi b/doc/coreutils.texi
> > +@item -z
> > +@itemx --zero
> > +@opindex -z
> > +@opindex --zero
> > +Separate output items with NUL characters.
> 
> s/Separate/Delimit/

Actually I copied that sentence from other sections.
I guess it's wrong there, too.  And probaby it makes
sense to replace it by a macro.

> > +@example
> > +$ id -Gn --zero
> > +users <NUL> devs
> 
> users <NUL> devs <NUL>

Good point!
 
> > diff --git a/tests/misc/id-zero.sh b/tests/misc/id-zero.sh
> 
> > +id root || fail=1
> 
> Is 'root' a guaranteed name?
> misc/chroot-credentials.sh since v7.4-16-gc45c51f assumes so.
> I'm still not convinced (and will look at addressing that separately).

Cygwin lacks a 'root' entry - but it also lacks getent ...

> > +# Create a nice list of users (ignoring errors) ...
> > +getent passwd | head -n 100 | cut -d: -f1 > users
> 
> test -s users || skip_ 'Failed to read the passwd database with getent'

... so this test is good; added.

> Also why so many (100). The test is a bit slow and you would get
> the same coverage from a couple of entries?

Actually that was to protect against "very-expensive" behavior on
bigger NIS installation. On normal installations, I haven't seen much
more than 30-40 users yet anyway.

The point was to get an entry with only 1 and with 2 or more groups.
I wouldn't go that far and add require_membership_in_two_groups_,
so maybe finding such entries with something like this would be better
(btw: can we rely on xargs to be there)?

  getent passwd | head -n50 | cut -d: -f1 \
    | xargs groups \
    | awk '{ x[NF] = $1; }
           END { for (u in x) print x[u]; }'

Anyway, I moved the tr(1) out of the loop, so we're saving a few
pipe creations.

> > +# id(1) should refuse --zero in default format.
> > +id --zero > out 2>err && fail=1
> > +compare /dev/null out || fail=1
> 
> > +grep 'option --zero not permitted in default format' err || fail=1
> 
> Better to:
> echo 'option ...' > exp-err
> compare exp-err err || fail=1

true, changed.
 
> > +printf "\n" > exp || framework_failure_
> 
> Minor point, but it's better to use '' quotes when not interpolating,
> here and throughout the test.

true, thanks.

> > +cp /dev/null  out || framework_failure_
> 
> :> out

yep, done.

Thanks for the review.
I'll provide a new version of the patch later (or tomorrow).

Have a nice day,
Berny

Message #64 received at 9987 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Bernhard Voelker <mail <at> bernhard-voelker.de>
Cc: 9987 <at> debbugs.gnu.org
Subject: Re: bug#9987: [PATCH] groups,id: add -0, --null option
Date: Thu, 19 Sep 2013 10:53:15 +0100

On 09/19/2013 08:56 AM, Bernhard Voelker wrote:
>>  On September 19, 2013 at 3:31 AM Pádraig Brady <P <at> draigBrady.com> wrote:

>>  Also why so many (100). The test is a bit slow and you would get
>>  the same coverage from a couple of entries?
> 
> Actually that was to protect against "very-expensive" behavior on
> bigger NIS installation. On normal installations, I haven't seen much
> more than 30-40 users yet anyway.

Yep that's still noticeable here.

> The point was to get an entry with only 1 and with 2 or more groups.
> I wouldn't go that far and add require_membership_in_two_groups_,
> so maybe finding such entries with something like this would be better

Oh I see. That need for 'root' definitely wasn't obvious and needs a comment.
With tests we have greater flexibility in that the common case can cater for
the coverage, while we can gracefully degrade the test on outliers.
So I'd avoid getent entirely and do:

# Add $USER to ensure we have at least one explicit entry
users="$u"
# Add 'root' to test multiple groups (as is usually the case)
id root >/dev/null 2>/dev/null && users="$USERS root"
# Add 'nobody' to test single group (as is usually the case)
id nobody >/dev/null 2>/dev/null && users="$USERS nobody"
# Add $users and '' (implicit $USER) to list to process
printf '%s\n' $users '' >> users || framework_failure_

> (btw: can we rely on xargs to be there)?

I see that xargs has been used since 2004 without issue.

thanks,
Pádraig.

Message #67 received at 9987 <at> debbugs.gnu.org (full text, mbox):

From: Bernhard Voelker <mail <at> bernhard-voelker.de>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 9987 <at> debbugs.gnu.org
Subject: Re: bug#9987: [PATCH] groups,id: add -0, --null option
Date: Sat, 21 Sep 2013 12:15:11 +0200

[Message part 1 (text/plain, inline)]

On 09/19/2013 11:53 AM, Pádraig Brady wrote:
> So I'd avoid getent entirely and do:

Good idea.

> # Add $USER to ensure we have at least one explicit entry
> users="$u"
> # Add 'root' to test multiple groups (as is usually the case)
> id root >/dev/null 2>/dev/null && users="$USERS root"
> # Add 'nobody' to test single group (as is usually the case)
> id nobody >/dev/null 2>/dev/null && users="$USERS nobody"
> # Add $users and '' (implicit $USER) to list to process
> printf '%s\n' $users '' >> users || framework_failure_

Interestingly, the situation is different here (openSUSE):
root is only in group root, and nobody is in the groups
nobody and nogroup.
However, I've added some users which seem to be typical
for being member of one or more groups.

  for u in root man postfix sshd nobody ; do
    id $u >/dev/null 2>&1 && users="$users $u"
  done

New patch attached.  Thanks!

Have a nice day,
Berny

[id-zero-option-v2.patch (text/x-patch, attachment)]

Message #70 received at 9987 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Bernhard Voelker <mail <at> bernhard-voelker.de>
Cc: 9987 <at> debbugs.gnu.org
Subject: Re: bug#9987: [PATCH] groups,id: add -0, --null option
Date: Sat, 21 Sep 2013 11:38:37 +0100

On 09/21/2013 11:15 AM, Bernhard Voelker wrote:
> diff --git a/tests/misc/id-zero.sh b/tests/misc/id-zero.sh

> +id root || fail=1

The 'root' name is guaranteed to be present,
so I'd avoid that line TBH.

Otherwise looks good.
Please Push.

thanks!
Pádraig.

Message #73 received at 9987 <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 9987 <at> debbugs.gnu.org, Bernhard Voelker <mail <at> bernhard-voelker.de>
Subject: Re: bug#9987: [PATCH] groups,id: add -0, --null option
Date: Sat, 21 Sep 2013 06:12:11 -0600

[Message part 1 (text/plain, inline)]

On 09/21/2013 04:38 AM, Pádraig Brady wrote:
> On 09/21/2013 11:15 AM, Bernhard Voelker wrote:
>> diff --git a/tests/misc/id-zero.sh b/tests/misc/id-zero.sh
> 
>> +id root || fail=1
> 
> The 'root' name is guaranteed to be present,

Not on Cygwin.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Message #78 received at 9987-done <at> debbugs.gnu.org (full text, mbox):

From: Bernhard Voelker <mail <at> bernhard-voelker.de>
To: Eric Blake <eblake <at> redhat.com>
Cc: 9987-done <at> debbugs.gnu.org,
 Pádraig Brady <P <at> draigBrady.com>
Subject: Re: bug#9987: [PATCH] groups,id: add -0, --null option
Date: Sat, 21 Sep 2013 14:19:17 +0200

On 09/21/2013 02:12 PM, Eric Blake wrote:
> On 09/21/2013 04:38 AM, Pádraig Brady wrote:
>> On 09/21/2013 11:15 AM, Bernhard Voelker wrote:
>>> diff --git a/tests/misc/id-zero.sh b/tests/misc/id-zero.sh
>>
>>> +id root || fail=1
>>
>> The 'root' name is guaranteed to be present,
> 
> Not on Cygwin.
> 


Okay, pushed with that change.
http://git.sv.gnu.org/cgit/coreutils.git/commit/?id=2a0d241f

Marking this bug as done.

Thanks & have a nice day,
Berny

Message #86 received at 9987 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: 9987 <at> debbugs.gnu.org, mail <at> bernhard-voelker.de, coreutils <at> tlinx.org
Subject: Re: bug#9987: [PATCH] groups,id: add -0, --null option
Date: Sat, 21 Sep 2013 14:37:53 +0100

On 09/21/2013 01:19 PM, Bernhard Voelker wrote:
> On 09/21/2013 02:12 PM, Eric Blake wrote:
>> On 09/21/2013 04:38 AM, Pádraig Brady wrote:
>>> On 09/21/2013 11:15 AM, Bernhard Voelker wrote:
>>>> diff --git a/tests/misc/id-zero.sh b/tests/misc/id-zero.sh
>>>
>>>> +id root || fail=1
>>>
>>> The 'root' name is guaranteed to be present,

s/is/is not/

>> Not on Cygwin.

Not anywhere in general, since 'root' is a arbitrary name.

> Okay, pushed with that change.
> http://git.sv.gnu.org/cgit/coreutils.git/commit/?id=2a0d241f
> 
> Marking this bug as done.

excellent.

thanks again,
Pádraig.

Message #89 received at 9987 <at> debbugs.gnu.org (full text, mbox):

From: Linda Walsh <coreutils <at> tlinx.org>
To: 9987 <at> debbugs.gnu.org
Subject: Re: bug#9987: closed (Re: bug#9987: [PATCH] groups, id: add -0,
 --null option)
Date: Sat, 21 Sep 2013 19:08:31 -0700

GNU bug Tracking System wrote:
> Your bug report
> 
> #9987: RFE: 'groups' command ADD command switches "-0", and "-1"
> 
> which was filed against the coreutils package, has been closed.
> 
> The explanation is attached below, along with your original report.
> If you require more details, please reply to 9987 <at> debbugs.gnu.org.
----
+ id accepts a new option: --zero (-z) to delimit the output entries by
+ a NUL instead of a white space character.
----

Curious: how many commands use "-z" to emit output with "0"
termination vs. something commands that have switches with "0" in them?

Message #92 received at 9987 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Linda Walsh <coreutils <at> tlinx.org>
Cc: 9987 <at> debbugs.gnu.org
Subject: Re: bug#9987: closed (Re: bug#9987: [PATCH] groups, id: add -0, --null
 option)
Date: Sun, 22 Sep 2013 03:22:19 +0100

On 09/22/2013 03:08 AM, Linda Walsh wrote:
> 
> 
> GNU bug Tracking System wrote:
>> Your bug report
>>
>> #9987: RFE: 'groups' command ADD command switches "-0", and "-1"
>>
>> which was filed against the coreutils package, has been closed.
>>
>> The explanation is attached below, along with your original report.
>> If you require more details, please reply to 9987 <at> debbugs.gnu.org.
> ----
> + id accepts a new option: --zero (-z) to delimit the output entries by
> + a NUL instead of a white space character.
> ----
> 
> Curious: how many commands use "-z" to emit output with "0"
> termination vs. something commands that have switches with "0" in them?

=== -0 ===
du (since v5.1.0) (POSIX doesn't mention -0)
env (since v8.0)
printenv (since v8.0)

=== -z ===
basename
dirname
id
join
readlink
realpath
shuf
sort
uniq

sed
grep

Message #95 received at 9987 <at> debbugs.gnu.org (full text, mbox):

From: Linda Walsh <coreutils <at> tlinx.org>
To: "PC!draig Brady" <P <at> draigBrady.com>
Cc: 9987 <at> debbugs.gnu.org
Subject: Re: bug#9987: closed (Re: bug#9987: [PATCH] groups, id: add -0,
 --null option)
Date: Sat, 21 Sep 2013 21:12:44 -0700

PC!draig Brady wrote:
> On 09/22/2013 03:08 AM, Linda Walsh wrote:
>>
>> GNU bug Tracking System wrote:
>>> Your bug report
>>>
>>> #9987: RFE: 'groups' command ADD command switches "-0", and "-1"
>>>
>>> which was filed against the coreutils package, has been closed.
>>>
>>> The explanation is attached below, along with your original report.
>>> If you require more details, please reply to 9987 <at> debbugs.gnu.org.
>> ----
>> + id accepts a new option: --zero (-z) to delimit the output entries by
>> + a NUL instead of a white space character.
>> ----
>>
>> Curious: how many commands use "-z" to emit output with "0"
>> termination vs. something commands that have switches with "0" in them?
> 
> === -0 ===
> du (since v5.1.0) (POSIX doesn't mention -0)
> env (since v8.0)
> printenv (since v8.0)
> 
> === -z ===
> basename
> dirname
> id
> join
> readlink
> realpath
> shuf
> sort
> uniq
> 
> sed
> grep
----
Some of those are fairly new.  At least id & join aren't even in my manpages yet.
The others using -z are fairly new. and the use of -z w/sed and grep is
suspect given their long options both refer to "null",

You left out find & xargs -- which ARE core utilities even if they are not
part of coreutils ;-).

It seems like -0 was the standard in the initial utils -- probably
because the word 'zero' in 17 out of hundreds of words for '0' in
other languages, but the arabic number system is used in almost all (if not all)
countries/languages.

It seems like someone, in the past few years, didn't follow the pre-existing
standard for such functionality.  Since "-z" is only a recent addition compared
to the non-locale-specific, "-0", wouldn't it make sense to try to use
"-0" where it doesn't conflict, since -z can't be used in tools like
grep as it indicates that input files are null terminated and uses -Z
for output.  I thought -0 might conflict somewhere, but it seems it
wouldn't conflict in any of the utilities.

Wouldn't it be a better choice -- especially considering it is international?

Or...  was there some reason why -z was chosen, as it makes the
switches less consistent and more confusing...(especially given the age of
the switches and that -Z has other meanings)...

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.