GNU bug report logs - #13028
inplace: new program

Previous Next

Package: coreutils;

Reported by: Reuben Thomas <rrt <at> sc3d.org>

Date: Thu, 29 Nov 2012 15:24:01 UTC

Severity: wishlist

To reply to this bug, email your comments to 13028 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-coreutils <at> gnu.org:
bug#13028; Package coreutils. (Thu, 29 Nov 2012 15:24:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Reuben Thomas <rrt <at> sc3d.org>:
New bug report received and forwarded. Copy sent to bug-coreutils <at> gnu.org. (Thu, 29 Nov 2012 15:24:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Reuben Thomas <rrt <at> sc3d.org>
To: bug-coreutils <bug-coreutils <at> gnu.org>
Subject: inplace
Date: Thu, 29 Nov 2012 15:20:47 +0000
[Message part 1 (text/plain, inline)]
On Fri, 14 May 2004 15:53:04 +0600 (YEKST), Victor Porton offered his handy
"inplace" script to coreutils, which runs a filter on a file in-place. A
couple of replies said there was no need for this as one could do in-place
editing with perl or sed, but I think that was misguided, as the point of
inplace is that it works with any filter. I just used it with xml_grep:

inplace "xml_grep --exclude 'ul[@class=\"toc\"]' \"\$IN\" > \"\$OUT\""
"$book.xhtml"

I attach an updated version of inplace with some bashisms and typos fixed.

-- 
http://rrt.sc3d.org
[Message part 2 (text/html, inline)]
[inplace (application/octet-stream, attachment)]

Information forwarded to bug-coreutils <at> gnu.org:
bug#13028; Package coreutils. (Thu, 29 Nov 2012 15:35:01 GMT) Full text and rfc822 format available.

Message #8 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ruediger Meier <sweet_f_a <at> gmx.de>
To: bug-coreutils <at> gnu.org
Subject: Re: bug#13028: inplace
Date: Thu, 29 Nov 2012 16:32:25 +0100
On Thursday 29 November 2012, Reuben Thomas wrote:
> On Fri, 14 May 2004 15:53:04 +0600 (YEKST), Victor Porton offered his
> handy "inplace" script to coreutils, which runs a filter on a file
> in-place. A couple of replies said there was no need for this as one
> could do in-place editing with perl or sed, but I think that was
> misguided, as the point of inplace is that it works with any filter.
> I just used it with xml_grep:
>
> inplace "xml_grep --exclude 'ul[@class=\"toc\"]' \"\$IN\" >
> \"\$OUT\"" "$book.xhtml"
>
> I attach an updated version of inplace with some bashisms and typos
> fixed.

You may also have a look at sponge from moreutils project
http://linux.die.net/man/1/sponge
http://joeyh.name/code/moreutils/


cu,
Rudi




Information forwarded to bug-coreutils <at> gnu.org:
bug#13028; Package coreutils. (Thu, 29 Nov 2012 15:38:01 GMT) Full text and rfc822 format available.

Message #11 received at 13028 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Reuben Thomas <rrt <at> sc3d.org>
Cc: 13028 <at> debbugs.gnu.org, "David A. Wheeler" <dwheeler <at> dwheeler.com>
Subject: Re: bug#13028: inplace
Date: Thu, 29 Nov 2012 15:35:09 +0000
On 11/29/2012 03:20 PM, Reuben Thomas wrote:
> On Fri, 14 May 2004 15:53:04 +0600 (YEKST), Victor Porton offered his handy "inplace" script to coreutils, which runs a filter on a file in-place. A couple of replies said there was no need for this as one could do in-place editing with perl or sed, but I think that was misguided, as the point of inplace is that it works with any filter. I just used it with xml_grep:
>
> inplace "xml_grep --exclude 'ul[@class=\"toc\"]' \"\$IN\" > \"\$OUT\"" "$book.xhtml"
>
> I attach an updated version of inplace with some bashisms and typos fixed.
>

I definitely think this is worthwhile.
In fact I mostly finished such a script here
that would handle all the atomic, backup, permissions, etc.:
http://lists.gnu.org/archive/html/bug-coreutils/2010-03/msg00213.html
See also: http://www.pixelbeat.org/docs/unix_file_replacement.html

Where to put such a script is an issue.
We were thinking of a contrib/ folder for higher level
scripts like this that could leverage coreutils/
Translations in the shell script was one thing that was
I was wondering about, which I need to look at.

Also I was wondering what to call it. rp, inplace, replace, ...
and in the meantime I noticed David Wheeler proposed much the same thing
with the "rewrite" util:
http://permalink.gmane.org/gmane.comp.standards.posix.austin.general/5348

rewrite is my favorite name so far.

I'll work on this script after I finish a pending
SELinux patchset I'm working on.

thanks,
Pádraig.




Information forwarded to bug-coreutils <at> gnu.org:
bug#13028; Package coreutils. (Thu, 29 Nov 2012 15:39:02 GMT) Full text and rfc822 format available.

Message #14 received at 13028 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Ruediger Meier <sweet_f_a <at> gmx.de>
Cc: 13028 <at> debbugs.gnu.org
Subject: Re: bug#13028: inplace
Date: Thu, 29 Nov 2012 15:36:48 +0000
On 11/29/2012 03:32 PM, Ruediger Meier wrote:
> On Thursday 29 November 2012, Reuben Thomas wrote:
>> On Fri, 14 May 2004 15:53:04 +0600 (YEKST), Victor Porton offered his
>> handy "inplace" script to coreutils, which runs a filter on a file
>> in-place. A couple of replies said there was no need for this as one
>> could do in-place editing with perl or sed, but I think that was
>> misguided, as the point of inplace is that it works with any filter.
>> I just used it with xml_grep:
>>
>> inplace "xml_grep --exclude 'ul[@class=\"toc\"]' \"\$IN\" >
>> \"\$OUT\"" "$book.xhtml"
>>
>> I attach an updated version of inplace with some bashisms and typos
>> fixed.
>
> You may also have a look at sponge from moreutils project
> http://linux.die.net/man/1/sponge
> http://joeyh.name/code/moreutils/

sponge is good, but doesn't quite cut it.
See point 6 at: http://www.pixelbeat.org/docs/unix_file_replacement.html

thanks,
Pádraig.




Information forwarded to bug-coreutils <at> gnu.org:
bug#13028; Package coreutils. (Thu, 29 Nov 2012 19:06:02 GMT) Full text and rfc822 format available.

Message #17 received at 13028 <at> debbugs.gnu.org (full text, mbox):

From: Reuben Thomas <rrt <at> sc3d.org>
To: Pádraig Brady <P <at> draigbrady.com>
Cc: 13028 <at> debbugs.gnu.org, "David A. Wheeler" <dwheeler <at> dwheeler.com>
Subject: Re: bug#13028: inplace
Date: Thu, 29 Nov 2012 19:03:14 +0000
[Message part 1 (text/plain, inline)]
On 29 November 2012 15:35, Pádraig Brady <P <at> draigbrady.com> wrote:

>
> I definitely think this is worthwhile.
>

Great!


> Where to put such a script is an issue.
> We were thinking of a contrib/ folder for higher level
> scripts like this that could leverage coreutils/
> Translations in the shell script was one thing that was
> I was wondering about, which I need to look at.
>

Is there a reason it can't be shipped so it'll install as part of coreutils?


> rewrite is my favorite name so far.
>

It sounds better, but I'd favour "inplace" as it's the term used by most
utilities, like Perl and sed, that already provide such a built-in
facility, and is hence more easily discovered.


> I'll work on this script after I finish a pending
> SELinux patchset I'm working on.


Great!

-- 
http://rrt.sc3d.org
[Message part 2 (text/html, inline)]

Information forwarded to bug-coreutils <at> gnu.org:
bug#13028; Package coreutils. (Thu, 29 Nov 2012 19:19:01 GMT) Full text and rfc822 format available.

Message #20 received at 13028 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Reuben Thomas <rrt <at> sc3d.org>
Cc: 13028 <at> debbugs.gnu.org, "David A. Wheeler" <dwheeler <at> dwheeler.com>
Subject: Re: bug#13028: inplace
Date: Thu, 29 Nov 2012 19:16:22 +0000
On 11/29/2012 07:03 PM, Reuben Thomas wrote:
> On 29 November 2012 15:35, Pádraig Brady <P <at> draigbrady.com> wrote:
>
>>
>> I definitely think this is worthwhile.
>>
>
> Great!
>
>
>> Where to put such a script is an issue.
>> We were thinking of a contrib/ folder for higher level
>> scripts like this that could leverage coreutils/
>> Translations in the shell script was one thing that was
>> I was wondering about, which I need to look at.
>>
>
> Is there a reason it can't be shipped so it'll install as part of coreutils?

Well contrib/ in upstream to be installed in standard locations.

>> rewrite is my favorite name so far.
>>
>
> It sounds better, but I'd favour "inplace" as it's the term used by most
> utilities, like Perl and sed, that already provide such a built-in
> facility, and is hence more easily discovered.

Previous notes on naming:
http://lists.gnu.org/archive/html/bug-coreutils/2010-03/msg00238.html

"rewrite" is more descriptive as a verb as to what is actually happening.
We can finalize on the naming anyway when we have the implementation sorted.

thanks,
Pádraig.




Information forwarded to bug-coreutils <at> gnu.org:
bug#13028; Package coreutils. (Fri, 30 Nov 2012 13:54:02 GMT) Full text and rfc822 format available.

Message #23 received at 13028 <at> debbugs.gnu.org (full text, mbox):

From: "David A. Wheeler" <dwheeler <at> dwheeler.com>
To: P <at> draigBrady.com
Cc: 13028 <at> debbugs.gnu.org, rrt <at> sc3d.org
Subject: Re: bug#13028: inplace
Date: Fri, 30 Nov 2012 08:51:16 -0500 (EST)
> On 11/29/2012 03:20 PM, Reuben Thomas wrote:
> > On Fri, 14 May 2004 15:53:04 +0600 (YEKST), Victor Porton offered his handy "inplace" script to coreutils, which runs a filter on a file in-place...

Pádraig Brady <P <at> draigBrady.com> said:
> I definitely think this is worthwhile.
> In fact I mostly finished such a script here
> that would handle all the atomic, backup, permissions, etc.:
> http://lists.gnu.org/archive/html/bug-coreutils/2010-03/msg00213.html
> See also: http://www.pixelbeat.org/docs/unix_file_replacement.html
...
> Also I was wondering what to call it. rp, inplace, replace, ...
> and in the meantime I noticed David Wheeler proposed much the same thing
> with the "rewrite" util:
> http://permalink.gmane.org/gmane.comp.standards.posix.austin.general/5348
...
> I'll work on this script after I finish a pending
> SELinux patchset I'm working on.

I'm delighted to hear that people are working this... thank you!

A request: Please make sure you include single-letter flags for key options; that'll help get it into the POSIX spec.  People are more likely to use tools like these if their interface is "available everywhere", and I think getting it (eventually) into the POSIX spec would help make that happen.

Also, please maximally steal useful syntax/examples/ideas from:
 http://permalink.gmane.org/gmane.comp.standards.posix.austin.general/5348
My hope is that this in-place edit tool will just "do the job well".

--- David A. Wheeler




Information forwarded to bug-coreutils <at> gnu.org:
bug#13028; Package coreutils. (Mon, 16 May 2016 13:16:01 GMT) Full text and rfc822 format available.

Message #26 received at 13028 <at> debbugs.gnu.org (full text, mbox):

From: Reuben Thomas <rrt <at> sc3d.org>
To: Pádraig Brady <P <at> draigbrady.com>
Cc: 13028 <at> debbugs.gnu.org, "David A. Wheeler" <dwheeler <at> dwheeler.com>
Subject: Re: bug#13028: inplace
Date: Mon, 16 May 2016 14:15:10 +0100
[Message part 1 (text/plain, inline)]
On 29 November 2012 at 19:16, Pádraig Brady <P <at> draigbrady.com> wrote:

> On 11/29/2012 07:03 PM, Reuben Thomas wrote:
>
>> On 29 November 2012 15:35, Pádraig Brady <P <at> draigbrady.com> wrote:
>>
>>
>>> I definitely think this is worthwhile.
>>>
>>>
>> Great!
>>
>>
>> Where to put such a script is an issue.
>>> We were thinking of a contrib/ folder for higher level
>>> scripts like this that could leverage coreutils/
>>> Translations in the shell script was one thing that was
>>> I was wondering about, which I need to look at.
>>>
>>>
>> Is there a reason it can't be shipped so it'll install as part of
>> coreutils?
>>
>
> Well contrib/ in upstream to be installed in standard locations.
>
> rewrite is my favorite name so far.
>>>
>>>
>> It sounds better, but I'd favour "inplace" as it's the term used by most
>> utilities, like Perl and sed, that already provide such a built-in
>> facility, and is hence more easily discovered.
>>
>
> Previous notes on naming:
> http://lists.gnu.org/archive/html/bug-coreutils/2010-03/msg00238.html
>
> "rewrite" is more descriptive as a verb as to what is actually happening.
> We can finalize on the naming anyway when we have the implementation
> sorted.
>

​Did this get anywhere?

-- 
http://rrt.sc3d.org
[Message part 2 (text/html, inline)]

Information forwarded to bug-coreutils <at> gnu.org:
bug#13028; Package coreutils. (Mon, 16 May 2016 13:43:02 GMT) Full text and rfc822 format available.

Message #29 received at 13028 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Reuben Thomas <rrt <at> sc3d.org>
Cc: 13028 <at> debbugs.gnu.org, "David A. Wheeler" <dwheeler <at> dwheeler.com>
Subject: Re: bug#13028: inplace
Date: Mon, 16 May 2016 14:42:04 +0100
On 16/05/16 14:15, Reuben Thomas wrote:
> On 29 November 2012 at 19:16, Pádraig Brady <P <at> draigbrady.com> wrote:
>
>> On 11/29/2012 07:03 PM, Reuben Thomas wrote:
>>
>>> On 29 November 2012 15:35, Pádraig Brady <P <at> draigbrady.com> wrote:
>>>
>>>
>>>> I definitely think this is worthwhile.
>>>>
>>>>
>>> Great!
>>>
>>>
>>> Where to put such a script is an issue.
>>>> We were thinking of a contrib/ folder for higher level
>>>> scripts like this that could leverage coreutils/
>>>> Translations in the shell script was one thing that was
>>>> I was wondering about, which I need to look at.
>>>>
>>>>
>>> Is there a reason it can't be shipped so it'll install as part of
>>> coreutils?
>>>
>>
>> Well contrib/ in upstream to be installed in standard locations.
>>
>> rewrite is my favorite name so far.
>>>>
>>>>
>>> It sounds better, but I'd favour "inplace" as it's the term used by most
>>> utilities, like Perl and sed, that already provide such a built-in
>>> facility, and is hence more easily discovered.
>>>
>>
>> Previous notes on naming:
>> http://lists.gnu.org/archive/html/bug-coreutils/2010-03/msg00238.html
>>
>> "rewrite" is more descriptive as a verb as to what is actually happening.
>> We can finalize on the naming anyway when we have the implementation
>> sorted.
>>
>
> ​Did this get anywhere?

Nothing public unfortunately.

Pádraig




Information forwarded to bug-coreutils <at> gnu.org:
bug#13028; Package coreutils. (Mon, 16 May 2016 13:48:02 GMT) Full text and rfc822 format available.

Message #32 received at 13028 <at> debbugs.gnu.org (full text, mbox):

From: Reuben Thomas <rrt <at> sc3d.org>
To: Pádraig Brady <P <at> draigbrady.com>
Cc: 13028 <at> debbugs.gnu.org, "David A. Wheeler" <dwheeler <at> dwheeler.com>
Subject: Re: bug#13028: inplace
Date: Mon, 16 May 2016 14:47:23 +0100
[Message part 1 (text/plain, inline)]
On 16 May 2016 at 14:42, Pádraig Brady <P <at> draigbrady.com> wrote:

> On 16/05/16 14:15, Reuben Thomas wrote:
>
>>
>> ​Did this get anywhere?
>>
>
> Nothing public unfortunately.


​Are there difficulties one might be able to help with?

-- 
http://rrt.sc3d.org
[Message part 2 (text/html, inline)]

Information forwarded to bug-coreutils <at> gnu.org:
bug#13028; Package coreutils. (Mon, 16 May 2016 14:43:01 GMT) Full text and rfc822 format available.

Message #35 received at 13028 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Reuben Thomas <rrt <at> sc3d.org>
Cc: 13028 <at> debbugs.gnu.org, "David A. Wheeler" <dwheeler <at> dwheeler.com>
Subject: Re: bug#13028: inplace
Date: Mon, 16 May 2016 15:42:20 +0100
On 16/05/16 14:47, Reuben Thomas wrote:
> On 16 May 2016 at 14:42, Pádraig Brady <P <at> draigbrady.com> wrote:
>
>> On 16/05/16 14:15, Reuben Thomas wrote:
>>
>>>
>>> ​Did this get anywhere?
>>>
>>
>> Nothing public unfortunately.
>
>
> ​Are there difficulties one might be able to help with?

I just don't have the time at present to complete this.

I did implement ACID file replacement using POSIX APIs a while ago in:
https://github.com/pixelb/crudini
The commit messages there have details on fsync()ing requirements etc.

Implementation in C in coreutils could also use other APIs where available
like renameat2(..., RENAME_EXCHANGE) and exchangedata() etc.
I noticed XFS_IOC_SWAPEXT but that's currently hardcoded
to support only same sized replacements (for defragment apps).
I was thinking of providing a wrapper for exchangedata() in gnulib,
which could be leveraged where available or falling back to
the current POSIX APIs.

thanks,
Pádraig




Severity set to 'wishlist' from 'normal' Request was from Assaf Gordon <assafgordon <at> gmail.com> to control <at> debbugs.gnu.org. (Thu, 18 Oct 2018 22:58:01 GMT) Full text and rfc822 format available.

Changed bug title to 'inplace: new program' from 'inplace' Request was from Assaf Gordon <assafgordon <at> gmail.com> to control <at> debbugs.gnu.org. (Thu, 18 Oct 2018 22:58:01 GMT) Full text and rfc822 format available.

Information forwarded to bug-coreutils <at> gnu.org:
bug#13028; Package coreutils. (Mon, 12 Sep 2022 08:08:01 GMT) Full text and rfc822 format available.

Message #42 received at 13028 <at> debbugs.gnu.org (full text, mbox):

From: Reuben Thomas <rrt <at> sc3d.org>
To: Pádraig Brady <P <at> draigbrady.com>
Cc: 13028 <at> debbugs.gnu.org, "David A. Wheeler" <dwheeler <at> dwheeler.com>
Subject: Re: bug#13028: inplace
Date: Mon, 12 Sep 2022 09:07:36 +0100
[Message part 1 (text/plain, inline)]
On Mon, 16 May 2016 at 15:42, Pádraig Brady <P <at> draigbrady.com> wrote:

>
> I just don't have the time at present to complete this.
>
> I did implement ACID file replacement using POSIX APIs a while ago in:
> https://github.com/pixelb/crudini
> The commit messages there have details on fsync()ing requirements etc.
>
> Implementation in C in coreutils could also use other APIs where available
> like renameat2(..., RENAME_EXCHANGE) and exchangedata() etc.
> I noticed XFS_IOC_SWAPEXT but that's currently hardcoded
> to support only same sized replacements (for defragment apps).
> I was thinking of providing a wrapper for exchangedata() in gnulib,
> which could be leveraged where available or falling back to
> the current POSIX APIs.
>

Did you get any further? I was prompted to look up this thread by a
deprecation message for "tempfile" from Victor Porton's "inplace" script,
which I still use!

-- 
https://rrt.sc3d.org
[Message part 2 (text/html, inline)]

Information forwarded to bug-coreutils <at> gnu.org:
bug#13028; Package coreutils. (Mon, 12 Sep 2022 11:58:02 GMT) Full text and rfc822 format available.

Message #45 received at 13028 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Reuben Thomas <rrt <at> sc3d.org>
Cc: 13028 <at> debbugs.gnu.org, "David A. Wheeler" <dwheeler <at> dwheeler.com>
Subject: Re: bug#13028: inplace
Date: Mon, 12 Sep 2022 12:57:07 +0100
On 12/09/2022 09:07, Reuben Thomas wrote:
> On Mon, 16 May 2016 at 15:42, Pádraig Brady <P <at> draigbrady.com <mailto:P <at> draigbrady.com>> wrote:
> 
> 
>     I just don't have the time at present to complete this.
> 
>     I did implement ACID file replacement using POSIX APIs a while ago in:
>     https://github.com/pixelb/crudini <https://github.com/pixelb/crudini>
>     The commit messages there have details on fsync()ing requirements etc.
> 
>     Implementation in C in coreutils could also use other APIs where available
>     like renameat2(..., RENAME_EXCHANGE) and exchangedata() etc.
>     I noticed XFS_IOC_SWAPEXT but that's currently hardcoded
>     to support only same sized replacements (for defragment apps).
>     I was thinking of providing a wrapper for exchangedata() in gnulib,
>     which could be leveraged where available or falling back to
>     the current POSIX APIs.
> 
> 
> Did you get any further? I was prompted to look up this thread by a deprecation message for "tempfile" from Victor Porton's "inplace" script, which I still use!

Not yet unfortunately





This bug report was last modified 1 year and 198 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.