GNU bug report logs - #34700
rm refuses to remove files owned by the user, even in force mode

Previous Next

To reply to this bug, email your comments to 34700 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Nicolas Mailhot <nicolas.mailhot <at> laposte.net>
To: bug-coreutils <at> gnu.org
Subject: rm refuses to remove files owned by the user, even in force mode
Date: Fri, 01 Mar 2019 13:48:51 +0100

Hi,

For their own reasons, the Go maintainers have decided the user Go cache
will now be read-only.
https://github.com/golang/go/issues/27161#issuecomment-433098406

That means cleaning up cache artefacts with rm does not work anymore
https://github.com/golang/go/issues/30502

I understand the need to protect read only files (even though they're
supposed to have been created by a user decision, not by user-hostile
tools). However should not rm remove the files anyway in force (-f) mode
without an explicit chmod first? Do it without bothering me is why -f
exists after all.

$ rpm -q --whatprovides /usr/bin/rm
coreutils-8.30-9.fc30.x86_64

Best regards,

-- 
Nicolas Mailhot

Message #8 received at 34700 <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: Nicolas Mailhot <nicolas.mailhot <at> laposte.net>, 34700 <at> debbugs.gnu.org
Subject: Re: bug#34700: rm refuses to remove files owned by the user, even in
 force mode
Date: Fri, 1 Mar 2019 07:58:38 -0600

On 3/1/19 6:48 AM, Nicolas Mailhot wrote:
> Hi,
> 
> For their own reasons, the Go maintainers have decided the user Go cache
> will now be read-only.
> https://github.com/golang/go/issues/27161#issuecomment-433098406
> 
> That means cleaning up cache artefacts with rm does not work anymore
> https://github.com/golang/go/issues/30502
> 
> I understand the need to protect read only files (even though they're
> supposed to have been created by a user decision, not by user-hostile
> tools). However should not rm remove the files anyway in force (-f) mode
> without an explicit chmod first? Do it without bothering me is why -f
> exists after all.

The behavior of -f is specified by POSIX:
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/rm.html

and is specified to suppress prompts and avoid diagnostics on missing
files, but NOT a means to force deletion via changing directory
permissions.  For that, you'd need a new option, because we can't change
the long-specified meaning of -f without breaking scripts.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

Message #11 received at 34700 <at> debbugs.gnu.org (full text, mbox):

From: Nicolas Mailhot <nicolas.mailhot <at> laposte.net>
To: Eric Blake <eblake <at> redhat.com>, 34700 <at> debbugs.gnu.org
Subject: Re: bug#34700: rm refuses to remove files owned by the user, even
 in force mode
Date: Fri, 01 Mar 2019 15:14:43 +0100

Le vendredi 01 mars 2019 à 07:58 -0600, Eric Blake a écrit :
> 
> The behavior of -f is specified by POSIX:
> http://pubs.opengroup.org/onlinepubs/9699919799/utilities/rm.html
> 
> and is specified to suppress prompts and avoid diagnostics on missing
> files, but NOT a means to force deletion via changing directory
> permissions.  For that, you'd need a new option, because we can't
> change
> the long-specified meaning of -f without breaking scripts.

Well it seems people got spoiled by other OSes where read-only is not-
really-read-only-when-you-own-the-file (IIRC vi also ignores permissions
in force mode).

So could a really-force flag be added?

Regards,

-- 
Nicolas Mailhot

Message #14 received at 34700 <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: Nicolas Mailhot <nicolas.mailhot <at> laposte.net>, 34700 <at> debbugs.gnu.org
Subject: Re: bug#34700: rm refuses to remove files owned by the user, even in
 force mode
Date: Fri, 1 Mar 2019 08:46:22 -0600

On 3/1/19 8:14 AM, Nicolas Mailhot wrote:
> Le vendredi 01 mars 2019 à 07:58 -0600, Eric Blake a écrit :
>>
>> The behavior of -f is specified by POSIX:
>> http://pubs.opengroup.org/onlinepubs/9699919799/utilities/rm.html
>>
>> and is specified to suppress prompts and avoid diagnostics on missing
>> files, but NOT a means to force deletion via changing directory
>> permissions.  For that, you'd need a new option, because we can't
>> change
>> the long-specified meaning of -f without breaking scripts.
> 
> Well it seems people got spoiled by other OSes where read-only is not-
> really-read-only-when-you-own-the-file (IIRC vi also ignores permissions
> in force mode).
> 
> So could a really-force flag be added?

Remember, unlink() requires write permissions on the container
directory. If a directory is read-only, unlink() alone CANNOT remove a
file, no matter who owns it (and that is true for ALL operating
systems). Unlinking a read-only file from a writable directory is
different than attempting to unlink from a read-only directory.

If rm is taught a new option to temporarily chmod() a directory to be
writable, just to perform the unlink(), should it then chmod() the
directory back to its original state or leave it writable when complete?
 What do you do about races (now that you are asking for a sequence of 2
or 3 syscalls instead of one, you have to worry about interactions with
another process also trying to cause you TOCTTOU security bugs)?  And
since such an option would be new, where you have to wait for new-enough
rm to be available in your distro, vs. the fact that you can already now
portably do a two-step chmod before rm.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

Message #17 received at 34700 <at> debbugs.gnu.org (full text, mbox):

From: Bob Proulx <bob <at> proulx.com>
To: Nicolas Mailhot <nicolas.mailhot <at> laposte.net>
Cc: 34700 <at> debbugs.gnu.org
Subject: Re: bug#34700: rm refuses to remove files owned by the user, even in
 force mode
Date: Fri, 1 Mar 2019 23:18:42 -0700

Nicolas Mailhot wrote:
> For their own reasons, the Go maintainers have decided the user Go cache
> will now be read-only.
> https://github.com/golang/go/issues/27161#issuecomment-433098406

Not wise.

> That means cleaning up cache artefacts with rm does not work anymore
> https://github.com/golang/go/issues/30502

Users count upon non-writable directories to prevent files from being
deleted.  I am confident that changing rm to delete contents of
non-writable directories would produce bug reports.  And worse it
would have resulted in data loss in those cases.  Weigh data loss
against inconvenience intentionally created.

They have intentionally done this to prevent actions such as rm -rf on
the path.  That is the entire purpose of making directories read-only,
to prevent the contents from being removed or renamed.

However regardless of intentions and design if one really wants to
smash it then this is easily scripted.  No code modifications are
needed.

  #!/bin/sh
  chmod -R u+w $1
  rm -rf $1

Bob

Message #20 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Erik Auerswald <auerswal <at> unix-ag.uni-kl.de>
To: bug-coreutils <at> gnu.org
Subject: Re: bug#34700: rm refuses to remove files owned by the user, even in
 force mode
Date: Sat, 2 Mar 2019 18:53:20 +0100

Hi,

On 3/2/19 07:18, Bob Proulx wrote:
> Nicolas Mailhot wrote:
>> For their own reasons, the Go maintainers have decided the user Go cache
>> will now be read-only.
>> https://github.com/golang/go/issues/27161#issuecomment-433098406
>> That means cleaning up cache artefacts with rm does not work anymore
>> https://github.com/golang/go/issues/30502
> [...]
> However regardless of intentions and design if one really wants to
> smash it then this is easily scripted.  No code modifications are
> needed.
> 
>    #!/bin/sh
>    chmod -R u+w $1
>    rm -rf $1

To everyone considering the above "script": do not use it! It does not 
even guard against spaces in file names. Besides being dangerously 
buggy, it does not even solve the problem of deleting a file inside a 
read-only directory.

I would suggest people with specific directories that inhibit deletion 
of files inside although they should not (e.g. a "cache") to deliberatly 
change the permissions of said directories prior to deleting files 
inside. Using a script like the above, even without the basic mistakes 
in the script, is quite dangerous.

Thanks,
Erik

Message #23 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Bob Proulx <bob <at> proulx.com>
To: bug-coreutils <at> gnu.org
Subject: Re: bug#34700: rm refuses to remove files owned by the user, even in
 force mode
Date: Sat, 2 Mar 2019 12:31:41 -0700

Erik Auerswald wrote:
> Bob Proulx wrote:
> > However regardless of intentions and design if one really wants to
> > smash it then this is easily scripted.  No code modifications are
> > needed.
> > 
> >    #!/bin/sh
> >    chmod -R u+w $1
> >    rm -rf $1
> 
> To everyone considering the above "script": do not use it! It does not even
> guard against spaces in file names. Besides being dangerously buggy, it does
> not even solve the problem of deleting a file inside a read-only directory.

Obviously I typed that in extemporaneously on the spur of the moment.
I should have put an "untested" tag upon it.

But regardless of that it does not change the fact that the entire
purpose of read-only directories is to prevent removing and renaming
of files within them.

> I would suggest people with specific directories that inhibit deletion of
> files inside although they should not (e.g. a "cache") to deliberatly change
> the permissions of said directories prior to deleting files inside. Using a
> script like the above, even without the basic mistakes in the script, is
> quite dangerous.

I don't think we are in disagreement here.

Bob

Message #26 received at submit <at> debbugs.gnu.org (full text, mbox):

From: L A Walsh <coreutils <at> tlinx.org>
To: bug-coreutils <at> gnu.org
Subject: Re: bug#34700: rm refuses to remove files owned by the user, even
 in force mode
Date: Sun, 03 Mar 2019 00:40:46 -0800

On 3/2/2019 11:31 AM, Bob Proulx wrote:
> But regardless of that it does not change the fact that the entire
> purpose of read-only directories is to prevent removing and renaming
> of files within them.
>   
----
    But not by the user owning them.  I don't remember the last time I used
chmod -w to prevent myself from deleting all the files in a directory. 
I use
it to prevent other people from removing/renaming.  If I want to
give myself "pause", I'll set them to immutable (on linux) and chmod
read-only or system on windows -- both of those give some pause to the
standard flow.  

    But read-only and owned by me, takes about an extra 1-2 seconds as
I tack sudo on the front and run right over user perms.

>   
>> I would suggest people with specific directories that inhibit deletion of
>> files inside although they should not (e.g. a "cache") to deliberatly change
>> the permissions of said directories prior to deleting files inside. Using a
>> script like the above, even without the basic mistakes in the script, is
>> quite dangerous.
>>     
Yeah...I wouldn't do it, I'd write a script that invokes the app and
clears out the cache dir when the app exits if it bothered me enough.

Much better to let the computer do the repetitive deletions.  If I do it
manually, it increases the chances of me creating a problem the more often
I do it. 

Really -- scripts are much better at handling redundant/routine matters that
turn parts of my brain off.  OTOH, some people are better at redundant
detail
and don't suffer the same problems I would.  People are different.
>   

Message #29 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Erik Auerswald <auerswal <at> unix-ag.uni-kl.de>
To: bug-coreutils <at> gnu.org
Subject: Re: bug#34700: rm refuses to remove files owned by the user, even in
 force mode
Date: Sun, 3 Mar 2019 13:43:49 +0100

Hi,

On 3/3/19 09:40, L A Walsh wrote:
> On 3/2/2019 11:31 AM, Bob Proulx wrote:
>> But regardless of that it does not change the fact that the entire
>> purpose of read-only directories is to prevent removing and renaming
>> of files within them.
>>    
> ----
>      But not by the user owning them.

The rationale given by the Go developers is to prevent downloaded test
code to remove or alter files in the modules directory, not to prevent
the user from doing that.

> [...]
>>> I would suggest people with specific directories that inhibit deletion of
>>> files inside although they should not (e.g. a "cache") to deliberatly change
>>> the permissions of said directories prior to deleting files inside. Using a
>>> script like the above, even without the basic mistakes in the script, is
>>> quite dangerous.
>>>      
> Yeah...I wouldn't do it, I'd write a script that invokes the app and
> clears out the cache dir when the app exits if it bothered me enough.

The Go developers implemented "go clean -modcache" for that purpose.
https://github.com/golang/go/issues/27161#issuecomment-415213240
https://tip.golang.org/cmd/go/#hdr-Remove_object_files_and_cached_files

> Much better to let the computer do the repetitive deletions.  If I do it
> manually, it increases the chances of me creating a problem the more often
> I do it.
> 
> Really -- scripts are much better at handling redundant/routine matters that
> turn parts of my brain off.  OTOH, some people are better at redundant
> detail
> and don't suffer the same problems I would.  People are different.

I concur to let software handle repetitive tasks.

If cleaning the cache occurs seldom, manually performing the changes, or
better invoking an existing specialized program (or script) for this
specific cache seems to be better than circumventing the safety net in a
general purpose utility. Especially if this circumvention means
transparently changing access rights on a directory that is not
mentioned in the utility invocation.

If cleaning the cache occurs all the time, using "go cache -modcache"
(or whatever program is appropriate for the specific cache) should be
the routine used. If there is no specialized program provided yet, a
script could be developed for that purpose.

Thanks,
Erik

Message #32 received at submit <at> debbugs.gnu.org (full text, mbox):

From: L A Walsh <coreutils <at> tlinx.org>
To: Coreutils <bug-coreutils <at> gnu.org>
Cc: Erik Auerswald <auerswal <at> unix-ag.uni-kl.de>, nicolas.mailhot <at> laposte.net
Subject: Re: bug#34700: rm refuses to remove files owned by the user, even
 in force mode
Date: Sun, 03 Mar 2019 07:48:54 -0800

On 3/3/2019 4:43 AM, Erik Auerswald wrote:
> Hi,
>
> On 3/3/19 09:40, L A Walsh wrote:
>   
>> On 3/2/2019 11:31 AM, Bob Proulx wrote:
>>     
>>> But regardless of that it does not change the fact that the entire
>>> purpose of read-only directories is to prevent removing and renaming
>>> of files within them.
>>>    
>>>       
>> ----
>>      But not by the user owning them.
>>     
>
> The rationale given by the Go developers is to prevent downloaded test
> code to remove or alter files in the modules directory, not to prevent
> the user from doing that.
>   
----
    I'm not familiar w/go, or why the user wants to purge something
that sounded equivalent to a web-cache, vs. the local extension-source
storage.

> The Go developers implemented "go clean -modcache" for that purpose.
> https://github.com/golang/go/issues/27161#issuecomment-415213240
> https://tip.golang.org/cmd/go/#hdr-Remove_object_files_and_cached_files
>   
----
    If there is a utilities for what the user needs, why are they wanting
'rm' to do it?  Is it just that they don't know about the util, or
maybe  the
util has constraints on its use to delete the source if the cached version
is deleted?  I don't know.  I do know that Google isn't known for following
common sense standards and like to force its own way on people, so I don't
know why the util isn't "working" for the original poster. 
> I concur to let software handle repetitive tasks.
----
    That's one of the reasons I got into computer SW development to
begin with.  I wanted to be able to program the computer to automatically
do the things I'd normally have to do manually.


> If cleaning the cache occurs all the time, using "go cache -modcache"
> (or whatever program is appropriate for the specific cache) should be
> the routine used. If there is no specialized program provided yet, a
> script could be developed for that purpose.
>   
----
    Well, that's might be the rub -- the current cache-cleaning program may
not be doing things "when" and "how" the user wants (I don't know).  Even
if there is a specialized program that provides one way to clean a cache,
that doesn't mean it works for the original user.  Another possibility is
that they might want to delete only files meeting some specific criteria. If
the provided tool doesn't do what they want or does "too much", they may
want to have their own way of cleaning things up.

    I've had more than one process that existed and worked well for me
before
google got involved.  My over-active imagination can easily come up with
situations and possibilities where the user might prefer their own controls
in place.

Cheers,
Linda

Message #35 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Nicolas Mailhot <nicolas.mailhot <at> laposte.net>
To: L A Walsh <coreutils <at> tlinx.org>
Cc: Erik Auerswald <auerswal <at> unix-ag.uni-kl.de>,
 Coreutils <bug-coreutils <at> gnu.org>
Subject: Re: bug#34700: rm refuses to remove files owned by the user, even in
 force mode
Date: Mon, 04 Mar 2019 08:42:58 +0100

Le 2019-03-03 16:48, L A Walsh a écrit :
> On 3/3/2019 4:43 AM, Erik Auerswald wrote:
>> Hi,
>> 
>> On 3/3/19 09:40, L A Walsh wrote:
>> 
>>> On 3/2/2019 11:31 AM, Bob Proulx wrote:
>>> 
>>>> But regardless of that it does not change the fact that the entire
>>>> purpose of read-only directories is to prevent removing and renaming
>>>> of files within them.
>>>> 
>>>> 
>>> ----
>>>      But not by the user owning them.
>>> 
>> 
>> The rationale given by the Go developers is to prevent downloaded test
>> code to remove or alter files in the modules directory, not to prevent
>> the user from doing that.
>> 
> ----
>     I'm not familiar w/go, or why the user wants to purge something
> that sounded equivalent to a web-cache, vs. the local extension-source
> storage.

The user (me) just wants to clean up files in his own home directory 
without special complications.

>> The Go developers implemented "go clean -modcache" for that purpose.
>> https://github.com/golang/go/issues/27161#issuecomment-415213240
>> https://tip.golang.org/cmd/go/#hdr-Remove_object_files_and_cached_files

And having files that need special commands to be modified is quite 
umconfortable

Not to mention the command may have been uninstalled leaving files on 
disk, is quite new, buggy, and full of unnexpected side effects (because 
it would have been too much to ask for a command that "just" cleans up 
the cache).

> ----
>     If there is a utilities for what the user needs, why are they 
> wanting
> 'rm' to do it?  Is it just that they don't know about the util, or
> maybe  the
> util has constraints on its use to delete the source if the cached 
> version
> is deleted?  I don't know.  I do know that Google isn't known for 
> following
> common sense standards and like to force its own way on people, so I 
> don't
> know why the util isn't "working" for the original poster.

It isn't "working". Besides the ro mode is not supposed to protect the 
files from users, it's supposed to protect the cache files from go unit 
tests, because google has moved go resources from plain directories to 
zip files, and code that attempts to manipulate zip files as if they 
were directories fails badly

As one of the Go devs commented in one of the tickets after arguing for 
the ro change "I'm surprised rm is not smart enough to do the right 
thing" (meaning remove everything when asked without requiring a 
permission change, since the user owns everything)

Regards,

-- 
Nicolas Mailhot

Message #38 received at 34700 <at> debbugs.gnu.org (full text, mbox):

From: Assaf Gordon <assafgordon <at> gmail.com>
To: Nicolas Mailhot <nicolas.mailhot <at> laposte.net>,
 L A Walsh <coreutils <at> tlinx.org>
Cc: auerswal <at> unix-ag.uni-kl.de, 34700 <at> debbugs.gnu.org
Subject: Re: bug#34700: rm refuses to remove files owned by the user, even in
 force mode
Date: Thu, 28 Mar 2019 12:27:20 -0600

tags 34700 notabug
severity 34700 wishlist
retitle: rm: add new --force option deal with read-only directories
stop

Hello,
As explained by several people in this thread,
This is not a bug in "rm -f", but the mandated behavior.

Bob and others provided work-arounds ( https://bugs.gnu.org/34700#17 ).

As for adding a new "--really-force" option
(https://bugs.gnu.org/34700#11) - I'm marking this as a wish-list
item.

-assaf

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.