GNU bug report logs -
#23904
Btrfs clone support in copy operations
Previous Next
Reported by: Paul Eggert <eggert <at> cs.ucla.edu>
Date: Wed, 6 Jul 2016 00:50:01 UTC
Severity: wishlist
Tags: patch
Done: Paul Eggert <eggert <at> cs.ucla.edu>
Bug is archived. No further changes may be made.
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 23904 in the body.
You can then email your comments to 23904 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#23904; Package
emacs.
(Wed, 06 Jul 2016 00:50:01 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
Paul Eggert <eggert <at> cs.ucla.edu>:
New bug report received and forwarded. Copy sent to
bug-gnu-emacs <at> gnu.org.
(Wed, 06 Jul 2016 00:50:01 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Tags: patch
The attached proposed patch to Emacs master builds on a suggestion by
Kieran Colford. Kieran, can you please comment on its two FIXMEs?
[0001-copy-file-now-uses-GNU-Linux-file-cloning.patch (text/x-patch, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#23904; Package
emacs.
(Wed, 06 Jul 2016 02:46:01 GMT)
Full text and
rfc822 format available.
Message #8 received at 23904 <at> debbugs.gnu.org (full text, mbox):
On 07/06/2016 03:49 AM, Paul Eggert wrote:
> The attached proposed patch to Emacs master builds on a suggestion by Kieran Colford.
IMHO this should use FICLONERANGE in a loop where you can
handle errors and use process_pending_signals as usual.
Dmitry
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#23904; Package
emacs.
(Wed, 06 Jul 2016 05:53:01 GMT)
Full text and
rfc822 format available.
Message #11 received at submit <at> debbugs.gnu.org (full text, mbox):
On Wed, Jul 06 2016, Dmitry Antipov wrote:
> On 07/06/2016 03:49 AM, Paul Eggert wrote:
>
>> The attached proposed patch to Emacs master builds on a suggestion
>> by Kieran Colford.
>
> IMHO this should use FICLONERANGE in a loop where you can
> handle errors and use process_pending_signals as usual.
This might also be a use case for sendfile(2).
Helmut
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#23904; Package
emacs.
(Wed, 06 Jul 2016 11:49:01 GMT)
Full text and
rfc822 format available.
Message #14 received at 23904 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 07/06/2016 04:45 AM, Dmitry Antipov wrote:
> IMHO this should use FICLONERANGE in a loop where you can
> handle errors and use process_pending_signals as usual.
Thanks, that suggestion should resolve the FIXME about how FICLONE
behaves when the output file is already larger than the input. However,
what should the clone chunk size be,each time through the loop? I don't
use btrfs so I don't know what a good size would be.
A downside of the suggestion is that the file copy won't be atomic.
However, the existing code isn't atomic, so this is nothing new.
Revised patch attached. Basically untested, since I don't use any file
systems that support FIOCLONERANGE. The patch still has FIXMEs for what
happens if the FIOCLONERANGE ioctl is interrupted by a signal, and for a
good clone chunk size (the patch guesses 1 GiB).
[0001-copy-file-now-uses-GNU-Linux-file-cloning.patch (text/x-patch, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#23904; Package
emacs.
(Wed, 06 Jul 2016 15:00:02 GMT)
Full text and
rfc822 format available.
Message #17 received at 23904 <at> debbugs.gnu.org (full text, mbox):
> From: Paul Eggert <eggert <at> cs.ucla.edu>
> Date: Wed, 6 Jul 2016 13:48:26 +0200
> Cc: sbaugh <at> catern.com, Kieran Colford <kieran <at> kcolford.com>
>
> Revised patch attached. Basically untested, since I don't use any file
> systems that support FIOCLONERANGE. The patch still has FIXMEs for what
> happens if the FIOCLONERANGE ioctl is interrupted by a signal, and for a
> good clone chunk size (the patch guesses 1 GiB).
Thanks.
However, I wonder: are we sure users will always want this
unconditionally, completely out of their control? Does 'cp' from
Coreutils do the same unconditionally, for example?
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#23904; Package
emacs.
(Wed, 06 Jul 2016 15:09:02 GMT)
Full text and
rfc822 format available.
Message #20 received at 23904 <at> debbugs.gnu.org (full text, mbox):
On 07/06/2016 04:58 PM, Eli Zaretskii wrote:
> are we sure users will always want this
> unconditionally, completely out of their control? Does 'cp' from
> Coreutils do the same unconditionally, for example?
No, coreutils cp has an option. As I recall it was put in soon after the
btrfs feature was added, and we were worried about bugs in btrfs. This
was several years ago. We're thinking of making cloning the default in
cp (on systems that support it) but haven't gotten around to it.
We could add another optional argument to copy-file as well, if there's
a significant need. I would omit such an option at first, though, as
that's simpler.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#23904; Package
emacs.
(Wed, 06 Jul 2016 15:28:02 GMT)
Full text and
rfc822 format available.
Message #23 received at 23904 <at> debbugs.gnu.org (full text, mbox):
> Cc: dmantipov <at> yandex.ru, 23904 <at> debbugs.gnu.org, sbaugh <at> catern.com,
> kieran <at> kcolford.com
> From: Paul Eggert <eggert <at> cs.ucla.edu>
> Date: Wed, 6 Jul 2016 17:08:33 +0200
>
> No, coreutils cp has an option. As I recall it was put in soon after the
> btrfs feature was added, and we were worried about bugs in btrfs. This
> was several years ago. We're thinking of making cloning the default in
> cp (on systems that support it) but haven't gotten around to it.
When 'cp' switches to using this by default, will it still have an
option to disable it? If so, I would think it will be too bold for
Emacs not provide such an option for users, even as opt-out.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#23904; Package
emacs.
(Wed, 06 Jul 2016 17:33:01 GMT)
Full text and
rfc822 format available.
Message #26 received at 23904 <at> debbugs.gnu.org (full text, mbox):
On 07/06/2016 05:26 PM, Eli Zaretskii wrote:
> When 'cp' switches to using this by default, will it still have an
> option to disable it?
Yes. It already has that option and we wouldn't remove it due to
backward-compatibility concerns, at least, not for many years. The
situation for Emacs is a bit different, as Emacs doesn't have the
backward-compatibility issues.
Reply sent
to
Paul Eggert <eggert <at> cs.ucla.edu>:
You have taken responsibility.
(Sun, 11 Sep 2016 02:17:02 GMT)
Full text and
rfc822 format available.
Notification sent
to
Paul Eggert <eggert <at> cs.ucla.edu>:
bug acknowledged by developer.
(Sun, 11 Sep 2016 02:17:02 GMT)
Full text and
rfc822 format available.
Message #31 received at 23904-done <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
I looked into this a bit more, and it appears that there's little point to
giving the user an option to clone or not clone when copying files, as the
cloning (or not cloning) can occur anyway.
We plan to release a coreutils version soon and to change the default to clone
after the release. For Emacs master the next release is far away, so there is
plenty of time to try out the FICLONE performance improvement. I installed the
attached two patches and am marking this bug report as done.
The second patch merely updates the documentation to discuss the issue; it is
logically independent of the first patch, since the issue can occur both with
and without the first patch.
[0001-copy-file-now-uses-GNU-Linux-file-cloning.patch (text/x-diff, attachment)]
[0002-Document-file-synchronization-issues.patch (text/x-diff, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#23904; Package
emacs.
(Sun, 11 Sep 2016 15:26:01 GMT)
Full text and
rfc822 format available.
Message #34 received at 23904 <at> debbugs.gnu.org (full text, mbox):
> Cc: dmantipov <at> yandex.ru, 23904-done <at> debbugs.gnu.org, sbaugh <at> catern.com,
> kieran <at> kcolford.com
> From: Paul Eggert <eggert <at> cs.ucla.edu>
> Date: Sat, 10 Sep 2016 19:16:39 -0700
>
>
> [1:text/plain Hide]
>
> I looked into this a bit more, and it appears that there's little point to
> giving the user an option to clone or not clone when copying files, as the
> cloning (or not cloning) can occur anyway.
>
> We plan to release a coreutils version soon and to change the default to clone
> after the release. For Emacs master the next release is far away, so there is
> plenty of time to try out the FICLONE performance improvement. I installed the
> attached two patches and am marking this bug report as done.
The first patch is fine with me.
> The second patch merely updates the documentation to discuss the issue; it is
> logically independent of the first patch, since the issue can occur both with
> and without the first patch.
I wonder why we need that text in the manual, it doesn't seem to
provide any useful actionable info, just general description of an
issue.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#23904; Package
emacs.
(Sun, 11 Sep 2016 17:18:01 GMT)
Full text and
rfc822 format available.
Message #37 received at 23904 <at> debbugs.gnu.org (full text, mbox):
Eli Zaretskii wrote:
> I wonder why we need that text in the manual, it doesn't seem to
> provide any useful actionable info, just general description of an
> issue.
The intent is to say that Emacs does not guarantee that its changes to files
might not survive power outages and the like. The new section briefly suggests
actions to take if this is of concern. Feel free to remove this stuff if you
think it so obvious or out-of-scope that it should not be in the manual.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#23904; Package
emacs.
(Sun, 11 Sep 2016 19:22:02 GMT)
Full text and
rfc822 format available.
Message #40 received at 23904 <at> debbugs.gnu.org (full text, mbox):
> Cc: dmantipov <at> yandex.ru, 23904 <at> debbugs.gnu.org, sbaugh <at> catern.com,
> kieran <at> kcolford.com
> From: Paul Eggert <eggert <at> cs.ucla.edu>
> Date: Sun, 11 Sep 2016 10:17:17 -0700
>
> Eli Zaretskii wrote:
> > I wonder why we need that text in the manual, it doesn't seem to
> > provide any useful actionable info, just general description of an
> > issue.
>
> The intent is to say that Emacs does not guarantee that its changes to files
> might not survive power outages and the like. The new section briefly suggests
> actions to take if this is of concern. Feel free to remove this stuff if you
> think it so obvious or out-of-scope that it should not be in the manual.
Does anyone else see this text as useful? I'm not going to remove it
if someone might find it helpful.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#23904; Package
emacs.
(Sun, 11 Sep 2016 22:45:02 GMT)
Full text and
rfc822 format available.
Message #43 received at 23904 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Come to think of it, the user manual already documents
write-region-inhibit-fsync, so it was a bit odd that the reference manual was
silent about it. I installed the attached further patch to document
write-region-inhibit-fsync in the reference manual, and to help the reference
manual avoid a dependency on the coreutils manual.
[0001-Remove-unnecessary-ref-to-coreutils-manual.patch (text/x-diff, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#23904; Package
emacs.
(Mon, 12 Sep 2016 02:37:01 GMT)
Full text and
rfc822 format available.
Message #46 received at 23904 <at> debbugs.gnu.org (full text, mbox):
> Cc: dmantipov <at> yandex.ru, 23904 <at> debbugs.gnu.org, sbaugh <at> catern.com,
> kieran <at> kcolford.com
> From: Paul Eggert <eggert <at> cs.ucla.edu>
> Date: Sun, 11 Sep 2016 15:44:45 -0700
>
> Come to think of it, the user manual already documents
> write-region-inhibit-fsync, so it was a bit odd that the reference manual was
> silent about it. I installed the attached further patch to document
> write-region-inhibit-fsync in the reference manual, and to help the reference
> manual avoid a dependency on the coreutils manual.
Thanks, I think now that subsection makes more sense.
But please remove the explicit @vindex entry, since @defvar already
creates one, and in the correct place.
bug archived.
Request was from
Debbugs Internal Request <help-debbugs <at> gnu.org>
to
internal_control <at> debbugs.gnu.org.
(Mon, 10 Oct 2016 11:24:03 GMT)
Full text and
rfc822 format available.
This bug report was last modified 7 years and 199 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.