GNU bug report logs - #19998
GREP_OPTIONS alternative?

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 19998 in the body.
You can then email your comments to 19998 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Christian Kujau <lists <at> nerdbynature.de>
To: bug-grep <at> gnu.org
Subject: GREP_OPTIONS alternative?
Date: Tue, 3 Mar 2015 21:55:18 -0800 (PST)

Hi,

much to my dismay I noticed that GREP_OPTIONS has been made obsolete. I 
usually kept the following in my .profile:

  export GREP_OPTIONS='--color=tty --devices=skip'

which was honored by GNU/grep and BSD/grep and ignored by e.g 
SUNWcsu/grep. Without GREP_OPTIONS, I have to put something like this in 
my .profile:

if grep --version 2>/dev/null | egrep -q 'GNU|BSD' 2>/dev/null; then
        alias   grep='grep   --color=auto --devices=skip'
fi


But that leaves out all the other grep variants, so what I really have in 
my .profile now is something like this:

if grep --version 2>/dev/null | egrep -q 'GNU|BSD' 2>/dev/null; then
        alias   grep='grep   --color=auto --devices=skip'
        alias  egrep='egrep  --color=auto --devices=skip'
        alias  fgrep='fgrep  --color=auto --devices=skip'
        alias  zgrep='zgrep  --color=auto --devices=skip'
        alias bzgrep='bzgrep --color=auto --devices=skip'
        alias xzgrep='xzgrep --color=auto --devices=skip'
fi

(and even that leaves out bzegrep, xzfgrep, etc.)

I suspect that I'm doing it wrong - but what's the alternative to have 
some nice grep defaults and still be portable across platforms?

Thanks,
Christian.
-- 
BOFH excuse #355:

Boredom in the Kernel.

Message #8 received at 19998 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Christian Kujau <lists <at> nerdbynature.de>, 19998 <at> debbugs.gnu.org
Subject: Re: bug#19998: GREP_OPTIONS alternative?
Date: Wed, 04 Mar 2015 14:43:41 -0800

On 03/03/2015 09:55 PM, Christian Kujau wrote:
> what's the alternative to have
> some nice grep defaults and still be portable across platforms?

You can put a script like this in $HOME/bin:

#! /bin/sh
export PATH=/usr/bin
exec grep --color=auto --devices=skip "$@"

and then prepend $HOME/bin to your PATH.

Message #11 received at 19998 <at> debbugs.gnu.org (full text, mbox):

From: Christian Kujau <lists <at> nerdbynature.de>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 19998 <at> debbugs.gnu.org
Subject: Re: bug#19998: GREP_OPTIONS alternative?
Date: Wed, 4 Mar 2015 17:04:06 -0800 (PST)

On Wed, 4 Mar 2015 at 14:43, Paul Eggert wrote:
> You can put a script like this in $HOME/bin:
> 
> #! /bin/sh
> export PATH=/usr/bin
> exec grep --color=auto --devices=skip "$@"
> 
> and then prepend $HOME/bin to your PATH.

Well, and I'd have to do this for every *grep variant, so instead of 
creating different aliases, I'm creating different scripts, which is an 
even greater change :-(

Is there really no other way?

I hope nobody removes PAGER, LESS or HISTCONTROL anytime soon :-\

Christian.
-- 
BOFH excuse #88:

Boss' kid fucked up the machine

Message #14 received at 19998 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Christian Kujau <lists <at> nerdbynature.de>
Cc: 19998 <at> debbugs.gnu.org
Subject: Re: bug#19998: GREP_OPTIONS alternative?
Date: Wed, 04 Mar 2015 18:29:01 -0800

Christian Kujau wrote:
> I'd have to do this for every *grep variant

No, not if the other variants invoke 'grep'.

Message #17 received at 19998 <at> debbugs.gnu.org (full text, mbox):

From: Norihiro Tanaka <noritnk <at> kcn.ne.jp>
To: Christian Kujau <lists <at> nerdbynature.de>
Cc: 19998 <at> debbugs.gnu.org, Paul Eggert <eggert <at> cs.ucla.edu>
Subject: Re: bug#19998: GREP_OPTIONS alternative?
Date: Fri, 06 Mar 2015 00:29:25 +0900

On Wed, 4 Mar 2015 17:04:06 -0800 (PST)
Christian Kujau <lists <at> nerdbynature.de> wrote:

> On Wed, 4 Mar 2015 at 14:43, Paul Eggert wrote:
> > You can put a script like this in $HOME/bin:
> > 
> > #! /bin/sh
> > export PATH=/usr/bin
> > exec grep --color=auto --devices=skip "$@"
> > 
> > and then prepend $HOME/bin to your PATH.
> 
> Well, and I'd have to do this for every *grep variant, so instead of 
> creating different aliases, I'm creating different scripts, which is an 
> even greater change :-(
> 
> Is there really no other way?
> 
> I hope nobody removes PAGER, LESS or HISTCONTROL anytime soon :-\
> 
> Christian.
> -- 
> BOFH excuse #88:
> 
> Boss' kid fucked up the machine
> 
> 

Please think to use grep -F and grep -E instead of fgrep and egrep.
For others, you can use following script, or customize grep's source
code and rebuild it to change default.

--
mv /bin/grep /bin/grep.bin
cat <<EOF >/bin/grep
#! /bin/sh
exec /bin/grep.bin --color=auto --devices=skip "$@"
EOF
chmod a+x /bin/grep
--


--
diff -ru grep-2.21.orig/src/grep.c grep-2.21/src/grep.c
--- grep-2.21.orig/src/grep.c   2014-11-09 19:00:37.000000000 +0000
+++ grep-2.21/src/grep.c        2015-03-05 15:26:59.000000000 +0000
@@ -72,7 +72,7 @@
 static bool suppress_errors;

 /* If nonzero, use color markers.  */
-static int color_option;
+static int color_option = 2;

 /* Show only the part of a line matching the expression. */
 static bool only_matching;
@@ -391,7 +391,7 @@
     READ_COMMAND_LINE_DEVICES,
     READ_DEVICES,
     SKIP_DEVICES
-  } devices = READ_COMMAND_LINE_DEVICES;
+  } devices = SKIP_DEVICES;

 static bool grepfile (int, char const *, bool, bool);
 static bool grepdesc (int, bool);
--

Message #20 received at 19998 <at> debbugs.gnu.org (full text, mbox):

From: Christian Kujau <lists <at> nerdbynature.de>
To: Norihiro Tanaka <noritnk <at> kcn.ne.jp>
Cc: 19998 <at> debbugs.gnu.org, Paul Eggert <eggert <at> cs.ucla.edu>
Subject: Re: bug#19998: GREP_OPTIONS alternative?
Date: Thu, 5 Mar 2015 11:24:56 -0800 (PST)

On Fri, 6 Mar 2015 at 00:29, Norihiro Tanaka wrote:
> Please think to use grep -F and grep -E instead of fgrep and egrep.
> For others, you can use following script, or customize grep's source
> code and rebuild it to change default.

OK, that (and the bits below) seem to confirm that there's no simple 
alternative to GREP_OPTIONS and one has to make serious changes to mimic 
that behaviour. Oh well...

Thanks for your input though.

Christian.

> 
> --
> mv /bin/grep /bin/grep.bin
> cat <<EOF >/bin/grep
> #! /bin/sh
> exec /bin/grep.bin --color=auto --devices=skip "$@"
> EOF
> chmod a+x /bin/grep
> --
> 
> 
> --
> diff -ru grep-2.21.orig/src/grep.c grep-2.21/src/grep.c
> --- grep-2.21.orig/src/grep.c   2014-11-09 19:00:37.000000000 +0000
> +++ grep-2.21/src/grep.c        2015-03-05 15:26:59.000000000 +0000
> @@ -72,7 +72,7 @@
>  static bool suppress_errors;
> 
>  /* If nonzero, use color markers.  */
> -static int color_option;
> +static int color_option = 2;
> 
>  /* Show only the part of a line matching the expression. */
>  static bool only_matching;
> @@ -391,7 +391,7 @@
>      READ_COMMAND_LINE_DEVICES,
>      READ_DEVICES,
>      SKIP_DEVICES
> -  } devices = READ_COMMAND_LINE_DEVICES;
> +  } devices = SKIP_DEVICES;
> 
>  static bool grepfile (int, char const *, bool, bool);
>  static bool grepdesc (int, bool);
> --
> 
> 

-- 
BOFH excuse #119:

evil hackers from Serbia.

Message #25 received at 19998-done <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Christian Kujau <lists <at> nerdbynature.de>
Cc: 19998-done <at> debbugs.gnu.org
Subject: Re: bug#19998: GREP_OPTIONS alternative?
Date: Wed, 11 Mar 2015 11:15:59 -0700

[Message part 1 (text/plain, inline)]

On 03/05/2015 11:24 AM, Christian Kujau wrote:
> that (and the bits below) seem to confirm that there's no simple
> alternative to GREP_OPTIONS

I don't see why the script solution is not considered 'simple'. Anyway, 
it's worth documenting the script alternative, so I installed the 
attached into the manual and am marking the bug as done.

[0001-doc-give-a-script-wrapper-example.patch (text/x-patch, attachment)]

Message #28 received at 19998-done <at> debbugs.gnu.org (full text, mbox):

From: Christian Kujau <lists <at> nerdbynature.de>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 19998-done <at> debbugs.gnu.org
Subject: Re: bug#19998: GREP_OPTIONS alternative?
Date: Wed, 11 Mar 2015 14:11:58 -0700 (PDT)

On Wed, 11 Mar 2015 at 11:15, Paul Eggert wrote:
> On 03/05/2015 11:24 AM, Christian Kujau wrote:
> > that (and the bits below) seem to confirm that there's no simple
> > alternative to GREP_OPTIONS
> 
> I don't see why the script solution is not considered 'simple'.

Hm, I don't understand how the script solution _can_ be considered 
'simple', but maybe I haven't made it clear enough. With GREP_OPTIONS, one 
only had to put a single line in .profile, which was honored by all the 
GNU grep variants that supported GREP_OPTIONS.

   export GREP_OPTIONS="...."

Without GREP_OPTIONS, one has to

1) Write a script for each grep variant (fgrep, zgrep, egrep, etc.)
2) put $HOME/bin in $PATH

I'd consider this much more complicated, especially 1) - either write a 
script for eeach variant or write a single script that pays attention on 
how it's called and put symlinks in $HOME/bin.

Also: on systems that do _not_ support these options, I cannot use this 
setup. To elaborate on this a bit more: I usually have a fairly static set 
of .dotfiles that I'd like to install in $HOME on a new machine. For 
instance, I'd set HISTTIMEFORMAT which is honored by newer versions of
'bash' but maybe the system does not have a bash installed and then this
variable is just ignored, no harm done. Same with GREP_OPTIONS - if 
GNU/grep was installed, the variable would be honored, other grep
implementations might choose to ignore it. If $HOME/bin/grep is really 
"grep --devices=skip", a the Solaris version of grep would bail with
"unrecognized option".

Of course, I understand the flip side of the coin: if GREP_OPTIONS gets 
set, maybe even system wide, shell scripts may behave differently. But 
that's true for every environment variable. If PATH omits /bin:/usr/bin, 
scripts maye behave very differently. Same for LC_ALL or or something 
else. Even GNU/tar has TAR_OPTIONS, so I don't really understand why 
GREP_OPTIONS is being dropped, just because some script broke because it 
didn't pay attention to environment variables.

> Anyway, it's
> worth documenting the script alternative, so I installed the attached into the
> manual and am marking the bug as done.

I didn't even expect that a mail the mailing list would open a formal bug 
report :-)

Christian.
-- 
BOFH excuse #277:

Your Flux Capacitor has gone bad.

Message #31 received at 19998-done <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Christian Kujau <lists <at> nerdbynature.de>
Cc: 19998-done <at> debbugs.gnu.org
Subject: Re: bug#19998: GREP_OPTIONS alternative?
Date: Wed, 11 Mar 2015 15:33:15 -0700

Christian Kujau wrote:
> Without GREP_OPTIONS, one has to
>
> 1) Write a script for each grep variant (fgrep, zgrep, egrep, etc.)
> 2) put $HOME/bin in $PATH

No, you don't need to write a script for each grep variant.  You need to write a 
script only for 'grep'.  That's because fgrep etc. invoke grep.

> Even GNU/tar has TAR_OPTIONS

Yes, and on my list of things to do is to remove TAR_OPTIONS as well, for the 
same reasons that GREP_OPTIONS should be removed.  (You have been warned. :-) 
It's too error-prone to have fundamental utilities' behavior depend on settings 
of random environment variables.

Message #34 received at 19998-done <at> debbugs.gnu.org (full text, mbox):

From: Christian Kujau <lists <at> nerdbynature.de>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 19998-done <at> debbugs.gnu.org
Subject: Re: bug#19998: GREP_OPTIONS alternative?
Date: Wed, 11 Mar 2015 15:53:35 -0700 (PDT)

On Wed, 11 Mar 2015 at 15:33, Paul Eggert wrote:
> > 1) Write a script for each grep variant (fgrep, zgrep, egrep, etc.)
> > 2) put $HOME/bin in $PATH
> 
> No, you don't need to write a script for each grep variant.  You need to write
> a script only for 'grep'.  That's because fgrep etc. invoke grep.

One might think so, but it doesn't work. On a Debian/Linux or a 
SuSE Linux system, /bin/*grep are not symlinks to /bin/grep but
seperate executables. Even zgrep (a shell script) rewrites itself
to /usr/bin/bin and thus $HOME/bin/grep is never called.

> It's too error-prone to have fundamental utilities' behavior depend on
> settings of random environment variables.

Why is GREP_OPTIONS random and LESS or ZIPOPT or GZIP are not?

Christian.


$ cat $HOME/bin/grep
#!/bin/sh
export PATH=/usr/bin:/bin
exec grep --line-number "$@"

$ unset GREP_OPTIONS
$ export PATH=$HOME/bin:$PATH

$ echo hello | grep ll
1:hello

$ echo hello | fgrep ll
hello
-- 
BOFH excuse #143:

had to use hammer to free stuck disk drive heads.

Message #37 received at 19998-done <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Christian Kujau <lists <at> nerdbynature.de>
Cc: 19998-done <at> debbugs.gnu.org
Subject: Re: bug#19998: GREP_OPTIONS alternative?
Date: Wed, 11 Mar 2015 17:58:42 -0700

[Message part 1 (text/plain, inline)]

Christian Kujau wrote:
> On a Debian/Linux or a
> SuSE Linux system,/bin/*grep are not symlinks to /bin/grep but
> seperate executables.

On my Ubuntu 14.10 system, fgrep lives in /bin and is a separate executable 
(actually a shell script), but it invokes 'grep'.  Likewise for Fedora 21, where 
fgrep lives in /usr/bin.  So what you write doesn't necessarily contradict what 
I said.

On the other hand, now that I've tested it, I see that when I type 'fgrep' Bash 
invokes it as '/bin/fgrep', which surprises me and which defeats the purpose of 
having 'fgrep' look at $0.  I installed the attached patch, which should fix that.

> Why is GREP_OPTIONS random and LESS or ZIPOPT or GZIP are not?

For starters, none of the other variables are used by standard utilities. 
Admittedly some judgment is needed here.  As LESS is merely a user-interface 
thing it's less worrisome.  I don't know what ZIPOPT is.  GZIP is worrisome and 
probably should go -- an attacker can use GZIP to cause 'gzip' to remove 
/etc/passwd, for example.  So it looks like I should add removing GZIP to my 
list of things to do too....

[0001-egrep-fgrep-just-use-what-s-in-PATH.patch (text/x-patch, attachment)]

Message #40 received at 19998 <at> debbugs.gnu.org (full text, mbox):

From: Santiago Ruano Rincón <santiago <at> debian.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 19998 <at> debbugs.gnu.org
Subject: Re: bug#19998: GREP_OPTIONS alternative?
Date: Thu, 12 Mar 2015 09:04:56 +0100

[Message part 1 (text/plain, inline)]

Hi,

El 11/03/15 a las 17:58, Paul Eggert escribió:
... 
> On the other hand, now that I've tested it, I see that when I type 'fgrep'
> Bash invokes it as '/bin/fgrep', which surprises me and which defeats the
> purpose of having 'fgrep' look at $0.  I installed the attached patch, which
> should fix that.
...

Then, please consider the attached patch, to clean src/Makefile.am up.

This shouldn't belong to this topic, but is there any reason to call
this wrapper with a shell other than sh (I've set it to /bin/sh in
Debian)

Cheers,

Santiago

[grep-Makefile-wrapper.patch (text/x-diff, attachment)]

Message #43 received at 19998-done <at> debbugs.gnu.org (full text, mbox):

From: Christian Kujau <lists <at> nerdbynature.de>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 19998-done <at> debbugs.gnu.org
Subject: Re: bug#19998: GREP_OPTIONS alternative?
Date: Fri, 13 Mar 2015 13:27:55 -0700 (PDT)

On Wed, 11 Mar 2015 at 17:58, Paul Eggert wrote:
> On my Ubuntu 14.10 system, fgrep lives in /bin and is a separate executable
> (actually a shell script), but it invokes 'grep'.  Likewise for Fedora 21,
> where fgrep lives in /usr/bin.  So what you write doesn't necessarily
> contradict what I said.

While it invokes "grep", it does not seem to invoke my $HOME/bin/grep, 
thus it will not honor the grep options set in $HOME/bin/grep.

> On the other hand, now that I've tested it, I see that when I type 'fgrep'
> Bash invokes it as '/bin/fgrep', which surprises me and which defeats the
> purpose of having 'fgrep' look at $0.  I installed the attached patch, which
> should fix that.

I hope that make is clear now why a helper script gets increasingly more 
complex if not complicated than a single environment variable.

>> Why is GREP_OPTIONS random and LESS or ZIPOPT or GZIP are not?
> 
> For starters, none of the other variables are used by standard utilities.

I'd consider gzip, zip, or less pretty "standard". I'd cite more standard 
utilities, but I don't want to put out any more ideas here...

> Admittedly some judgment is needed here.  As LESS is merely a user-interface
> thing it's less worrisome.  I don't know what ZIPOPT is.

From zip(1):

> The environment variable ZIPOPT can be used to change the default
> options. 

So, pretty much what GREP_OPTIONS is^Wused to be.

> GZIP is worrisome and probably should go -- an attacker can use GZIP to 
> cause 'gzip' to remove /etc/passwd, for example.  So it looks like I 

An "attacker" can set $PATH to /tmp and do stuff too. If an attacker can 
modify a user's environment, all is lost anyway.

> should add removing GZIP to my list of things to do too....

I was afraid you'd say something like this :-\

Christian.
-- 
BOFH excuse #277:

Your Flux Capacitor has gone bad.

Message #46 received at 19998-done <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Christian Kujau <lists <at> nerdbynature.de>
Cc: 19998-done <at> debbugs.gnu.org
Subject: Re: bug#19998: GREP_OPTIONS alternative?
Date: Fri, 13 Mar 2015 19:11:36 -0700

Christian Kujau wrote:
> An "attacker" can set $PATH to /tmp and do stuff too.

Sure, but that's well-known and standardized and it's easy (and expected) for 
administrative applications to sanitize PATH.  The problem comes when we have 
lots of mysterious little environment variables any of which can wreak havoc.

Message #49 received at 19998-done <at> debbugs.gnu.org (full text, mbox):

From: Christian Kujau <lists <at> nerdbynature.de>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 19998-done <at> debbugs.gnu.org
Subject: Re: bug#19998: GREP_OPTIONS alternative?
Date: Fri, 13 Mar 2015 21:13:19 -0700 (PDT)

On Fri, 13 Mar 2015 at 19:11, Paul Eggert wrote:
> > An "attacker" can set $PATH to /tmp and do stuff too.
> 
> Sure, but that's well-known and standardized and it's easy (and expected) for
> administrative applications to sanitize PATH.  The problem comes when we have

s/PATH/TMPDIR/ - or LD_PRELOAD or LD_LIBRARY_PATH, etc. All "well known" 
and "potentially dangerous" if not cared for.

I relalize of course  that you won't change your mind about GREP_OPTIONS, 
but I'm a bit surpised that such a visible change in userspace was done 
w/o any consideration of the users of said feature. Oh well...

Christian.
-- 
BOFH excuse #338:

old inkjet cartridges emanate barium-based fumes

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.