GNU bug report logs - #51857
cross-filesystem copying broken on macOS with coreutils >= 9.0

Previous Next

Package: coreutils;

Reported by: Sudhip Nashi <sudhipnashi <at> icloud.com>

Date: Mon, 15 Nov 2021 05:02:02 UTC

Severity: normal

Done: Paul Eggert <eggert <at> cs.ucla.edu>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 51857 in the body.
You can then email your comments to 51857 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-coreutils <at> gnu.org:
bug#51857; Package coreutils. (Mon, 15 Nov 2021 05:02:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Sudhip Nashi <sudhipnashi <at> icloud.com>:
New bug report received and forwarded. Copy sent to bug-coreutils <at> gnu.org. (Mon, 15 Nov 2021 05:02:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Sudhip Nashi <sudhipnashi <at> icloud.com>
To: bug-coreutils <at> gnu.org
Cc: me <at> cameronkatri.com
Subject: cross-filesystem copying broken on macOS with coreutils >= 9.0 
Date: Sun, 14 Nov 2021 22:02:33 -0600
Hello,

Cross-filesystem copying seems to have been broken in the latest coreutils release on macOS. Running a command like ‘cp /usr/bin/clear /tmp/test’ appears to return successfully, but if one analyzes /tmp/test, it’s filled with NULL characters. However, copying works fine when the source and destination file are on the same filesystem. Do you know what might be causing this?

Thanks in advance,
Sudhip Nashi



Information forwarded to bug-coreutils <at> gnu.org:
bug#51857; Package coreutils. (Mon, 15 Nov 2021 15:10:02 GMT) Full text and rfc822 format available.

Message #8 received at 51857 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Sudhip Nashi <sudhipnashi <at> icloud.com>, 51857 <at> debbugs.gnu.org
Cc: me <at> cameronkatri.com
Subject: Re: bug#51857: cross-filesystem copying broken on macOS with
 coreutils >= 9.0
Date: Mon, 15 Nov 2021 15:09:25 +0000
On 15/11/2021 04:02, Sudhip Nashi via GNU coreutils Bug Reports wrote:
> Hello,
> 
> Cross-filesystem copying seems to have been broken in the latest coreutils release on macOS. Running a command like ‘cp /usr/bin/clear /tmp/test’ appears to return successfully, but if one analyzes /tmp/test, it’s filled with NULL characters. However, copying works fine when the source and destination file are on the same filesystem. Do you know what might be causing this?
> 
> Thanks in advance,
> Sudhip Nashi
> 
> 

What are the source and dest file system types?
Could you send the output of `sudo dtruss cp /usr/bin/clear /tmp/test`?
I suspect SEEK_DATA may have issues on nacOS,
as usage of that is new in coreutils 9.0.

thanks,
Pádraig




Information forwarded to bug-coreutils <at> gnu.org:
bug#51857; Package coreutils. (Mon, 15 Nov 2021 16:01:01 GMT) Full text and rfc822 format available.

Message #11 received at 51857 <at> debbugs.gnu.org (full text, mbox):

From: Cameron Katri <me <at> cameronkatri.com>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: Sudhip Nashi <sudhipnashi <at> icloud.com>, 51857 <at> debbugs.gnu.org
Subject: Re: bug#51857: cross-filesystem copying broken on macOS with
 coreutils >= 9.0
Date: Mon, 15 Nov 2021 10:48:19 -0500
> On Nov 15, 2021, at 10:09 AM, Pádraig Brady <P <at> draigBrady.com> wrote:
> 
> On 15/11/2021 04:02, Sudhip Nashi via GNU coreutils Bug Reports wrote:
>> Hello,
>> Cross-filesystem copying seems to have been broken in the latest coreutils release on macOS. Running a command like ‘cp /usr/bin/clear /tmp/test’ appears to return successfully, but if one analyzes /tmp/test, it’s filled with NULL characters. However, copying works fine when the source and destination file are on the same filesystem. Do you know what might be causing this?
>> Thanks in advance,
>> Sudhip Nashi
> 
> What are the source and dest file system types?
> Could you send the output of `sudo dtruss cp /usr/bin/clear /tmp/test`?
> I suspect SEEK_DATA may have issues on nacOS,
> as usage of that is new in coreutils 9.0.
> 
> thanks,
> Pádraig

Here yo go:

sudo dtruss ./src/cp /usr/bin/clear /tmp/test
SYSCALL(args) 		 = return
access("/AppleInternal/XBS/.isChrooted\0", 0x0, 0x0)		 = -1 Err#2
bsdthread_register(0x1AEC802C8, 0x1AEC802BC, 0x4000)		 = 1073742303 0
shm_open(0x1AEB48F55, 0x0, 0xFFFFFFFFB821D000)		 = 3 0
fstat64(0x3, 0x16DDC2070, 0x0)		 = 0 0
mmap(0x0, 0x4000, 0x1, 0x40001, 0x3, 0x0)		 = 0x102170000 0
close(0x3)		 = 0 0
ioctl(0x2, 0x4004667A, 0x16DDC211C)		 = 0 0
mprotect(0x1021A0000, 0x4000, 0x0)		 = 0 0
mprotect(0x1021AC000, 0x4000, 0x0)		 = 0 0
mprotect(0x1021B0000, 0x4000, 0x0)		 = 0 0
mprotect(0x1021BC000, 0x4000, 0x0)		 = 0 0
mprotect(0x1021C0000, 0x4000, 0x0)		 = 0 0
mprotect(0x1021CC000, 0x4000, 0x0)		 = 0 0
mprotect(0x102174000, 0x90, 0x1)		 = 0 0
mprotect(0x102174000, 0x90, 0x3)		 = 0 0
mprotect(0x102174000, 0x90, 0x1)		 = 0 0
mprotect(0x10217C000, 0x4000, 0x1)		 = 0 0
mprotect(0x1021D0000, 0x90, 0x1)		 = 0 0
mprotect(0x1021D0000, 0x90, 0x3)		 = 0 0
mprotect(0x1021D0000, 0x90, 0x1)		 = 0 0
mprotect(0x102174000, 0x90, 0x3)		 = 0 0
mprotect(0x102174000, 0x90, 0x1)		 = 0 0
mprotect(0x10217C000, 0x4000, 0x3)		 = 0 0
mprotect(0x10217C000, 0x4000, 0x1)		 = 0 0
objc_bp_assist_cfg_np(0x1AEB103C0, 0x8000000000201048, 0x0)		 = -1 Err#5
issetugid(0x0, 0x0, 0x0)		 = 0 0
getentropy(0x16DDC1EC8, 0x20, 0x0)		 = 0 0
getentropy(0x16DDC1F18, 0x40, 0x0)		 = 0 0
getpid(0x0, 0x0, 0x0)		 = 4661 0
stat64("/AppleInternal\0", 0x16DDC2680, 0x0)		 = -1 Err#2
csops_audittoken(0x1235, 0x7, 0x16DDC21B0)		 = 0 0
proc_info(0x2, 0x1235, 0xD)		 = 64 0
csops_audittoken(0x1235, 0x7, 0x16DDC2270)		 = 0 0
sysctlbyname(kern.osvariant_status, 0x15, 0x16DDC26E8, 0x16DDC26E0, 0x0)		 = 0 0
csops(0x1235, 0x0, 0x16DDC270C)		 = 0 0
geteuid(0x0, 0x0, 0x0)		 = 0 0
getuid(0x0, 0x0, 0x0)		 = 0 0
sysctl([CTL_KERN, 14, 1, 4661, 0, 0] (4), 0x16DDC0C00, 0x16DDC0BE8, 0x0, 0x0)		 = 0 0
gettid(0x16DDC0EE0, 0x16DDC0EE4, 0x0)		 = -1 Err#3
geteuid(0x0, 0x0, 0x0)		 = 0 0
getegid(0x0, 0x0, 0x0)		 = 0 0
csops(0x1235, 0x0, 0x16DDC1C24)		 = 0 0
gettid(0x16DDC0EB0, 0x16DDC0EB4, 0x0)		 = -1 Err#3
geteuid(0x0, 0x0, 0x0)		 = 0 0
getegid(0x0, 0x0, 0x0)		 = 0 0
mprotect(0x102068000, 0x100000, 0x1)		 = 0 0
open_nocancel("/usr/share/locale/en_US.UTF-8/LC_COLLATE\0", 0x0, 0x0)		 = 3 0
fcntl_nocancel(0x3, 0x3, 0x0)		 = 0 0
getrlimit(0x1008, 0x16DDC2F98, 0x0)		 = 0 0
fstat64(0x3, 0x16DDC2F10, 0x0)		 = 0 0
read_nocancel(0x3, "1.1A\n\0", 0x1000)		 = 2086 0
close_nocancel(0x3)		 = 0 0
open_nocancel("/usr/share/locale/en_US.UTF-8/LC_CTYPE\0", 0x0, 0x0)		 = 3 0
fcntl_nocancel(0x3, 0x3, 0x0)		 = 0 0
fstat64(0x3, 0x16DDC3040, 0x0)		 = 0 0
fstat64(0x3, 0x16DDC2E30, 0x0)		 = 0 0
lseek(0x3, 0x0, 0x1)		 = 0 0
lseek(0x3, 0x0, 0x0)		 = 0 0
read_nocancel(0x3, "RuneMagAUTF-8\0", 0x1000)		 = 4096 0
read_nocancel(0x3, "\0", 0x1000)		 = 4096 0
read_nocancel(0x3, "\0", 0x1000)		 = 4096 0
read_nocancel(0x3, "\0", 0x1000)		 = 4096 0
read_nocancel(0x3, "\0", 0x1000)		 = 4096 0
read_nocancel(0x3, "\0", 0x1000)		 = 4096 0
read_nocancel(0x3, "\0", 0x1000)		 = 4096 0
read_nocancel(0x3, "@\004\211\0", 0xF5D0)		 = 62928 0
close_nocancel(0x3)		 = 0 0
open_nocancel("/usr/share/locale/en_US.UTF-8/LC_MONETARY\0", 0x0, 0x0)		 = 3 0
fstat64(0x3, 0x16DDC3060, 0x0)		 = 0 0
read_nocancel(0x3, "USD \n$\n.\n,\n3;3\n\n-\n2\n2\n1\n0\n1\n0\n1\n1\n(\0", 0x22)		 = 34 0
close_nocancel(0x3)		 = 0 0
open_nocancel("/usr/share/locale/en_US.UTF-8/LC_NUMERIC\0", 0x0, 0x0)		 = 3 0
fstat64(0x3, 0x16DDC3060, 0x0)		 = 0 0
read_nocancel(0x3, ".\n,\n3;3\n@$\b\0", 0x8)		 = 8 0
close_nocancel(0x3)		 = 0 0
open_nocancel("/usr/share/locale/en_US.UTF-8/LC_TIME\0", 0x0, 0x0)		 = 3 0
fstat64(0x3, 0x16DDC3070, 0x0)		 = 0 0
read_nocancel(0x3, "Jan\nFeb\nMar\nApr\nMay\nJun\nJul\nAug\nSep\nOct\nNov\nDec\nJanuary\nFebruary\nMarch\nApril\nMay\nJune\nJuly\nAugust\nSeptember\nOctober\nNovember\nDecember\nSun\nMon\nTue\nWed\nThu\nFri\nSat\nSunday\nMonday\nTuesday\nWednesday\nThursday\nFriday\nSaturday\n%H:%M:%S\n%m/%d/%Y\n%a %b %e %X %Y\nAM\nP", 0x179)		 = 377 0
close_nocancel(0x3)		 = 0 0
open_nocancel("/usr/share/locale/en_US.UTF-8/LC_MESSAGES/LC_MESSAGES\0", 0x0, 0x0)		 = 3 0
fstat64(0x3, 0x16DDC3070, 0x0)		 = 0 0
read_nocancel(0x3, "^[yYsS].*\n^[nN].*\n(\0", 0x12)		 = 18 0
close_nocancel(0x3)		 = 0 0
geteuid(0x0, 0x0, 0x0)		 = 0 0
stat64("/tmp/test\0", 0x16DDC36C0, 0x0)		 = 0 0
fstatat64(0xFFFFFFFFFFFFFFFE, 0x16DDC3C21, 0x16DDC2BA0)		 = 0 0
fstatat64(0xFFFFFFFFFFFFFFFE, 0x16DDC3C30, 0x16DDC2B10)		 = 0 0
open("/usr/bin/clear\0", 0x0, 0x0)		 = 3 0
fstat64(0x3, 0x16DDC2C30, 0x0)		 = 0 0
open("/tmp/test\0", 0x401, 0x0)		 = 4 0
fstat64(0x4, 0x16DDC2CC0, 0x0)		 = 0 0
fstat64(0x4, 0x16DDC2D50, 0x0)		 = 0 0
fcntl(0x3, 0x32, 0x16DDC3200)		 = 0 0
fcntl(0x4, 0x32, 0x16DDC2E00)		 = 0 0
unlink("/private/tmp/test\0", 0x0, 0x0)		 = 0 0
clonefileat(0xFFFFFFFFFFFFFFFE, 0x16DDC3200, 0xFFFFFFFFFFFFFFFE)		 = -1 Err#18
open("/private/tmp/test\0", 0x601, 0x81ED)		 = 5 0
close(0x5)		 = 0 0
open("/private/tmp/test\0", 0x2, 0x0)		 = 5 0
dup2(0x5, 0x4, 0x0)		 = 4 0
close(0x5)		 = 0 0
fchmod(0x4, 0x81ED, 0x0)		 = 0 0
fchown(0x4, 0x0, 0x0)		 = 0 0
futimes(0x4, 0x16DDC2DE0, 0x0)		 = 0 0
sysctl([CTL_HW, 7, 0, 0, 0, 0] (2), 0x207EC4068, 0x16DDC2A30, 0x0, 0x0)		 = 0 0
lseek(0x3, 0x0, 0x4)		 = -1 Err#6
ftruncate(0x4, 0x1D770, 0x0)		 = 0 0
close(0x4)		 = 0 0
close(0x3)		 = 0 0
lseek(0x0, 0x0, 0x1)		 = 6741 0
lseek(0x0, 0x0, 0x1)		 = 6741 0
lseek(0x0, 0x1A55, 0x0)		 = 6741 0
close_nocancel(0x0)		 = 0 0
close_nocancel(0x1)		 = 0 0
close_nocancel(0x2)		 = 0 0





Information forwarded to bug-coreutils <at> gnu.org:
bug#51857; Package coreutils. (Mon, 15 Nov 2021 17:34:02 GMT) Full text and rfc822 format available.

Message #14 received at 51857 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Cameron Katri <me <at> cameronkatri.com>, Pádraig Brady
 <P <at> draigBrady.com>
Cc: Sudhip Nashi <sudhipnashi <at> icloud.com>, 51857 <at> debbugs.gnu.org
Subject: Re: bug#51857: cross-filesystem copying broken on macOS with
 coreutils >= 9.0
Date: Mon, 15 Nov 2021 09:33:21 -0800
Is the source file on a ZFS file system by any chance? See my lseek 
comment below.

On 11/15/21 07:48, Cameron Katri via GNU coreutils Bug Reports wrote:

> stat64("/tmp/test\0", 0x16DDC36C0, 0x0)		 = 0 0
> fstatat64(0xFFFFFFFFFFFFFFFE, 0x16DDC3C21, 0x16DDC2BA0)		 = 0 0
> fstatat64(0xFFFFFFFFFFFFFFFE, 0x16DDC3C30, 0x16DDC2B10)		 = 0 0
> open("/usr/bin/clear\0", 0x0, 0x0)		 = 3 0
> fstat64(0x3, 0x16DDC2C30, 0x0)		 = 0 0
> open("/tmp/test\0", 0x401, 0x0)		 = 4 0
> fstat64(0x4, 0x16DDC2CC0, 0x0)		 = 0 0
> fstat64(0x4, 0x16DDC2D50, 0x0)		 = 0 0
> fcntl(0x3, 0x32, 0x16DDC3200)		 = 0 0
> fcntl(0x4, 0x32, 0x16DDC2E00)		 = 0 0
> unlink("/private/tmp/test\0", 0x0, 0x0)		 = 0 0

What's causing this use of "/private/tmp"? I don't see that in the GNU 
cp source code. Can you put a breakpoint on clonefileat and see what's 
calling it and what its arguments are?

> clonefileat(0xFFFFFFFFFFFFFFFE, 0x16DDC3200, 0xFFFFFFFFFFFFFFFE)		 = -1 Err#18
> open("/private/tmp/test\0", 0x601, 0x81ED)		 = 5 0
> close(0x5)		 = 0 0
> open("/private/tmp/test\0", 0x2, 0x0)		 = 5 0
> dup2(0x5, 0x4, 0x0)		 = 4 0
> close(0x5)		 = 0 0
> fchmod(0x4, 0x81ED, 0x0)		 = 0 0
> fchown(0x4, 0x0, 0x0)		 = 0 0
> futimes(0x4, 0x16DDC2DE0, 0x0)		 = 0 0
> sysctl([CTL_HW, 7, 0, 0, 0, 0] (2), 0x207EC4068, 0x16DDC2A30, 0x0, 0x0)		 = 0 0
> lseek(0x3, 0x0, 0x4)		 = -1 Err#6

That lseek call looks like OpenZFS bug 11900 
<https://github.com/openzfs/zfs/issues/11900>. If you're using ZFS, the 
bug really should be fixed in your ZFS implementation as it can affect 
programs other than coreutils and there's no easy workaround (other than 
to disable efficient copying). Is this something you can look into, or 
ask someone with macOS and/or ZFS expertise to look into? For more, see 
<https://bugs.gnu.org/51433>.

> ftruncate(0x4, 0x1D770, 0x0)		 = 0 0
> close(0x4)		 = 0 0
> close(0x3)		 = 0 0




Information forwarded to bug-coreutils <at> gnu.org:
bug#51857; Package coreutils. (Mon, 15 Nov 2021 17:41:01 GMT) Full text and rfc822 format available.

Message #17 received at 51857 <at> debbugs.gnu.org (full text, mbox):

From: Cameron Katri <me <at> cameronkatri.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: Sudhip Nashi <sudhipnashi <at> icloud.com>,
 Pádraig Brady <P <at> draigBrady.com>, 51857 <at> debbugs.gnu.org
Subject: Re: bug#51857: cross-filesystem copying broken on macOS with
 coreutils >= 9.0
Date: Mon, 15 Nov 2021 12:40:45 -0500
[Message part 1 (text/plain, inline)]
On Mon, Nov 15, 2021 at 09:33:21AM -0800, Paul Eggert wrote:
> Is the source file on a ZFS file system by any chance? See my lseek comment
> below.

No, this is one APFS (Apple File System).

> 
> On 11/15/21 07:48, Cameron Katri via GNU coreutils Bug Reports wrote:
> 
> > stat64("/tmp/test\0", 0x16DDC36C0, 0x0)		 = 0 0
> > fstatat64(0xFFFFFFFFFFFFFFFE, 0x16DDC3C21, 0x16DDC2BA0)		 = 0 0
> > fstatat64(0xFFFFFFFFFFFFFFFE, 0x16DDC3C30, 0x16DDC2B10)		 = 0 0
> > open("/usr/bin/clear\0", 0x0, 0x0)		 = 3 0
> > fstat64(0x3, 0x16DDC2C30, 0x0)		 = 0 0
> > open("/tmp/test\0", 0x401, 0x0)		 = 4 0
> > fstat64(0x4, 0x16DDC2CC0, 0x0)		 = 0 0
> > fstat64(0x4, 0x16DDC2D50, 0x0)		 = 0 0
> > fcntl(0x3, 0x32, 0x16DDC3200)		 = 0 0
> > fcntl(0x4, 0x32, 0x16DDC2E00)		 = 0 0
> > unlink("/private/tmp/test\0", 0x0, 0x0)		 = 0 0
> 
> What's causing this use of "/private/tmp"? I don't see that in the GNU cp
> source code. Can you put a breakpoint on clonefileat and see what's calling
> it and what its arguments are?

On macOS, `/tmp` is a symlink to `/private/tmp`.

> 
> > clonefileat(0xFFFFFFFFFFFFFFFE, 0x16DDC3200, 0xFFFFFFFFFFFFFFFE)		 = -1 Err#18
> > open("/private/tmp/test\0", 0x601, 0x81ED)		 = 5 0
> > close(0x5)		 = 0 0
> > open("/private/tmp/test\0", 0x2, 0x0)		 = 5 0
> > dup2(0x5, 0x4, 0x0)		 = 4 0
> > close(0x5)		 = 0 0
> > fchmod(0x4, 0x81ED, 0x0)		 = 0 0
> > fchown(0x4, 0x0, 0x0)		 = 0 0
> > futimes(0x4, 0x16DDC2DE0, 0x0)		 = 0 0
> > sysctl([CTL_HW, 7, 0, 0, 0, 0] (2), 0x207EC4068, 0x16DDC2A30, 0x0, 0x0)		 = 0 0
> > lseek(0x3, 0x0, 0x4)		 = -1 Err#6
> 
> That lseek call looks like OpenZFS bug 11900
> <https://github.com/openzfs/zfs/issues/11900>. If you're using ZFS, the bug
> really should be fixed in your ZFS implementation as it can affect programs
> other than coreutils and there's no easy workaround (other than to disable
> efficient copying). Is this something you can look into, or ask someone with
> macOS and/or ZFS expertise to look into? For more, see
> <https://bugs.gnu.org/51433>.
> 
> > ftruncate(0x4, 0x1D770, 0x0)		 = 0 0
> > close(0x4)		 = 0 0
> > close(0x3)		 = 0 0

-- 
Cameron Katri
Email: me <at> cameronkatri.com
PGP Fingerprint: 7D3B36CEA40FCC2181FB6DCDBAFFD97826540F1C
[signature.asc (application/pgp-signature, inline)]

Information forwarded to bug-coreutils <at> gnu.org:
bug#51857; Package coreutils. (Mon, 15 Nov 2021 20:52:01 GMT) Full text and rfc822 format available.

Message #20 received at 51857 <at> debbugs.gnu.org (full text, mbox):

From: Sudhip Nashi <sudhipnashi <at> icloud.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 51857 <at> debbugs.gnu.org, Pádraig Brady <P <at> draigbrady.com>,
 Cameron Katri <me <at> cameronkatri.com>
Subject: Re: bug#51857: cross-filesystem copying broken on macOS with
 coreutils >= 9.0
Date: Mon, 15 Nov 2021 12:37:51 -0600
> On Nov 15, 2021, at 11:33, Paul Eggert <eggert <at> cs.ucla.edu> wrote:
> 
> Is the source file on a ZFS file system by any chance? See my lseek comment below.
> 
>> On 11/15/21 07:48, Cameron Katri via GNU coreutils Bug Reports wrote:
>> 
>> stat64("/tmp/test\0", 0x16DDC36C0, 0x0)         = 0 0
>> fstatat64(0xFFFFFFFFFFFFFFFE, 0x16DDC3C21, 0x16DDC2BA0)         = 0 0
>> fstatat64(0xFFFFFFFFFFFFFFFE, 0x16DDC3C30, 0x16DDC2B10)         = 0 0
>> open("/usr/bin/clear\0", 0x0, 0x0)         = 3 0
>> fstat64(0x3, 0x16DDC2C30, 0x0)         = 0 0
>> open("/tmp/test\0", 0x401, 0x0)         = 4 0
>> fstat64(0x4, 0x16DDC2CC0, 0x0)         = 0 0
>> fstat64(0x4, 0x16DDC2D50, 0x0)         = 0 0
>> fcntl(0x3, 0x32, 0x16DDC3200)         = 0 0
>> fcntl(0x4, 0x32, 0x16DDC2E00)         = 0 0
>> unlink("/private/tmp/test\0", 0x0, 0x0)         = 0 0
> 
> What's causing this use of "/private/tmp"? I don't see that in the GNU cp source code. Can you put a breakpoint on clonefileat and see what's calling it and what its arguments are?
> 
>> clonefileat(0xFFFFFFFFFFFFFFFE, 0x16DDC3200, 0xFFFFFFFFFFFFFFFE)         = -1 Err#18
>> open("/private/tmp/test\0", 0x601, 0x81ED)         = 5 0
>> close(0x5)         = 0 0
>> open("/private/tmp/test\0", 0x2, 0x0)         = 5 0
>> dup2(0x5, 0x4, 0x0)         = 4 0
>> close(0x5)         = 0 0
>> fchmod(0x4, 0x81ED, 0x0)         = 0 0
>> fchown(0x4, 0x0, 0x0)         = 0 0
>> futimes(0x4, 0x16DDC2DE0, 0x0)         = 0 0
>> sysctl([CTL_HW, 7, 0, 0, 0, 0] (2), 0x207EC4068, 0x16DDC2A30, 0x0, 0x0)         = 0 0
>> lseek(0x3, 0x0, 0x4)         = -1 Err#6
> 
> That lseek call looks like OpenZFS bug 11900 <https://github.com/openzfs/zfs/issues/11900>. If you're using ZFS, the bug really should be fixed in your ZFS implementation as it can affect programs other than coreutils and there's no easy workaround (other than to disable efficient copying). Is this something you can look into, or ask someone with macOS and/or ZFS expertise to look into? For more, see <https://bugs.gnu.org/51433>.
> 
>> ftruncate(0x4, 0x1D770, 0x0)         = 0 0
>> close(0x4)         = 0 0
>> close(0x3)         = 0 0

Turns out lseek is broken (or at least works differently) on macOS as well (https://lists.gnu.org/archive/html/bug-gnulib/2018-09/msg00054.html). Funny coincidence! I’ll take a better look later this week if I can and try to see what the exact problem is.



Information forwarded to bug-coreutils <at> gnu.org:
bug#51857; Package coreutils. (Mon, 15 Nov 2021 21:34:01 GMT) Full text and rfc822 format available.

Message #23 received at 51857 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Cameron Katri <me <at> cameronkatri.com>
Cc: Sudhip Nashi <sudhipnashi <at> icloud.com>,
 Pádraig Brady <P <at> draigBrady.com>, 51857 <at> debbugs.gnu.org
Subject: Re: bug#51857: cross-filesystem copying broken on macOS with
 coreutils >= 9.0
Date: Mon, 15 Nov 2021 13:33:44 -0800
On 11/15/21 09:40, Cameron Katri wrote:

> No, this is one APFS (Apple File System).

OK, so ZFS is not involved.


>>> unlink("/private/tmp/test\0", 0x0, 0x0)		 = 0 0
>>
>> What's causing this use of "/private/tmp"? I don't see that in the GNU cp
>> source code. Can you put a breakpoint on clonefileat and see what's calling
>> it and what its arguments are?
> 
> On macOS, `/tmp` is a symlink to `/private/tmp`.

Fine, but why would 'cp' remove /private/tmp/test when you told it to 
copy to /tmp/test? I see no reason why it would expand the symlink by 
hand, nor why it would remove the destination file even if it calculated 
that /tmp/test and /private/tmp/test were the same file (it's not 
supposed to do that).

And why would 'cp' invoke clonefileat? Coreutils cp's source code does 
not mention clonefileat anywhere.

Something very odd is going on here.

Did you build vanilla coreutils 9.0 yourself? If so, what commands did 
you you use to build it, exactly? If not, who built coreutils and how 
did they configure and/or modify it? I worry that we're looking at a 
version of coreutils cp that has been modified somehow, or that you're 
dtrussing the wrong cp somehow.




Information forwarded to bug-coreutils <at> gnu.org:
bug#51857; Package coreutils. (Mon, 15 Nov 2021 21:50:02 GMT) Full text and rfc822 format available.

Message #26 received at 51857 <at> debbugs.gnu.org (full text, mbox):

From: Cameron Katri <me <at> cameronkatri.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: Sudhip Nashi <sudhipnashi <at> icloud.com>,
 Pádraig Brady <P <at> draigBrady.com>, 51857 <at> debbugs.gnu.org
Subject: Re: bug#51857: cross-filesystem copying broken on macOS with
 coreutils >= 9.0
Date: Mon, 15 Nov 2021 16:49:40 -0500
[Message part 1 (text/plain, inline)]
On Mon, Nov 15, 2021 at 01:33:44PM -0800, Paul Eggert wrote:
> On 11/15/21 09:40, Cameron Katri wrote:
> 
> Did you build vanilla coreutils 9.0 yourself? If so, what commands did you
> you use to build it, exactly? If not, who built coreutils and how did they
> configure and/or modify it? I worry that we're looking at a version of
> coreutils cp that has been modified somehow, or that you're dtrussing the
> wrong cp somehow.

I forgot that I had a patch to enable reflink on APFS, I just rebuilt a
vanilla coreutils with just ./configure && make and the issue persists.
Sorry about that, here is the correct dtruss:

cameron in Documents/coreutils-9.0/src at build
\> sudo dtruss ./cp /usr/bin/clear /tmp/test
SYSCALL(args)            = return
access("/AppleInternal/XBS/.isChrooted\0", 0x0, 0x0)             = -1 Err#2
bsdthread_register(0x1AEC802C8, 0x1AEC802BC, 0x4000)             = 1073742303 0
shm_open(0x1AEB48F55, 0x0, 0x4158000)            = 3 0
fstat64(0x3, 0x16BCA6130, 0x0)           = 0 0
mmap(0x0, 0x4000, 0x1, 0x40001, 0x3, 0x0)                = 0x104298000 0
close(0x3)               = 0 0
ioctl(0x2, 0x4004667A, 0x16BCA61DC)              = 0 0
mprotect(0x1042A4000, 0x4000, 0x0)               = 0 0
mprotect(0x1042B0000, 0x4000, 0x0)               = 0 0
mprotect(0x1042B4000, 0x4000, 0x0)               = 0 0
mprotect(0x1042C0000, 0x4000, 0x0)               = 0 0
mprotect(0x1042C4000, 0x4000, 0x0)               = 0 0
mprotect(0x1042D0000, 0x4000, 0x0)               = 0 0
mprotect(0x10429C000, 0x90, 0x1)                 = 0 0
mprotect(0x10429C000, 0x90, 0x3)                 = 0 0
mprotect(0x10429C000, 0x90, 0x1)                 = 0 0
mprotect(0x1042D4000, 0x4000, 0x1)               = 0 0
mprotect(0x1042D8000, 0x90, 0x1)                 = 0 0
mprotect(0x1042D8000, 0x90, 0x3)                 = 0 0
mprotect(0x1042D8000, 0x90, 0x1)                 = 0 0
mprotect(0x10429C000, 0x90, 0x3)                 = 0 0
mprotect(0x10429C000, 0x90, 0x1)                 = 0 0
mprotect(0x1042D4000, 0x4000, 0x3)               = 0 0
mprotect(0x1042D4000, 0x4000, 0x1)               = 0 0
objc_bp_assist_cfg_np(0x1AEB103C0, 0x8000000000201048, 0x0)              = -1 Err#5
issetugid(0x0, 0x0, 0x0)                 = 0 0
getentropy(0x16BCA5FF8, 0x20, 0x0)               = 0 0
getentropy(0x16BCA6048, 0x40, 0x0)               = 0 0
getpid(0x0, 0x0, 0x0)            = 91358 0
stat64("/AppleInternal\0", 0x16BCA6740, 0x0)             = -1 Err#2
csops_audittoken(0x164DE, 0x7, 0x16BCA6270)              = 0 0
proc_info(0x2, 0x164DE, 0xD)             = 64 0
csops_audittoken(0x164DE, 0x7, 0x16BCA6330)              = 0 0
sysctlbyname(kern.osvariant_status, 0x15, 0x16BCA67A8, 0x16BCA67A0, 0x0)                 = 0 0
csops(0x164DE, 0x0, 0x16BCA67CC)                 = 0 0
mprotect(0x104190000, 0x100000, 0x1)             = 0 0
open_nocancel("/usr/share/locale/en_US.UTF-8/LC_COLLATE\0", 0x0, 0x0)            = 3 0
fcntl_nocancel(0x3, 0x3, 0x0)            = 0 0
getrlimit(0x1008, 0x16BCA7178, 0x0)              = 0 0
fstat64(0x3, 0x16BCA70F0, 0x0)           = 0 0
read_nocancel(0x3, "1.1A\n\0", 0x1000)           = 2086 0
close_nocancel(0x3)              = 0 0
open_nocancel("/usr/share/locale/en_US.UTF-8/LC_CTYPE\0", 0x0, 0x0)              = 3 0
fcntl_nocancel(0x3, 0x3, 0x0)            = 0 0
fstat64(0x3, 0x16BCA7220, 0x0)           = 0 0
fstat64(0x3, 0x16BCA7010, 0x0)           = 0 0
lseek(0x3, 0x0, 0x1)             = 0 0
lseek(0x3, 0x0, 0x0)             = 0 0
read_nocancel(0x3, "RuneMagAUTF-8\0", 0x1000)            = 4096 0
read_nocancel(0x3, "\0", 0x1000)                 = 4096 0
read_nocancel(0x3, "\0", 0x1000)                 = 4096 0
read_nocancel(0x3, "\0", 0x1000)                 = 4096 0
read_nocancel(0x3, "\0", 0x1000)                 = 4096 0
read_nocancel(0x3, "\0", 0x1000)                 = 4096 0
read_nocancel(0x3, "\0", 0x1000)                 = 4096 0
read_nocancel(0x3, "@\004\211\0", 0xF5D0)                = 62928 0
close_nocancel(0x3)              = 0 0
open_nocancel("/usr/share/locale/en_US.UTF-8/LC_MONETARY\0", 0x0, 0x0)           = 3 0
fstat64(0x3, 0x16BCA7240, 0x0)           = 0 0
read_nocancel(0x3, "USD \n$\n.\n,\n3;3\n\n-\n2\n2\n1\n0\n1\n0\n1\n1\n(\0", 0x22)                 = 34 0
close_nocancel(0x3)              = 0 0
open_nocancel("/usr/share/locale/en_US.UTF-8/LC_NUMERIC\0", 0x0, 0x0)            = 3 0
fstat64(0x3, 0x16BCA7240, 0x0)           = 0 0
read_nocancel(0x3, ".\n,\n3;3\n@$\b\0", 0x8)             = 8 0
close_nocancel(0x3)              = 0 0
open_nocancel("/usr/share/locale/en_US.UTF-8/LC_TIME\0", 0x0, 0x0)               = 3 0
fstat64(0x3, 0x16BCA7250, 0x0)           = 0 0
read_nocancel(0x3, "Jan\nFeb\nMar\nApr\nMay\nJun\nJul\nAug\nSep\nOct\nNov\nDec\nJanuary\nFebruary\nMarch\nApril\nMay\nJune\nJuly\nAugust\nSeptember\nOctober\nNovember\nDecember\nSun\nMon\nTue\nWed\nThu\nFri\nSat\nSunday\nMonday\nTuesday\nWednesday\nThursday\nFriday\nSaturday\n%H:%M:%S\n%m/%d/%Y\n%a %b %e %X %Y\nAM\nP", 0x179)            = 377 0
close_nocancel(0x3)              = 0 0
open_nocancel("/usr/share/locale/en_US.UTF-8/LC_MESSAGES/LC_MESSAGES\0", 0x0, 0x0)               = 3 0
fstat64(0x3, 0x16BCA7250, 0x0)           = 0 0
read_nocancel(0x3, "^[yYsS].*\n^[nN].*\n(\0", 0x12)              = 18 0
close_nocancel(0x3)              = 0 0
geteuid(0x0, 0x0, 0x0)           = 0 0
stat64("/tmp/test\0", 0x16BCA7720, 0x0)          = 0 0
fstatat64(0xFFFFFFFFFFFFFFFE, 0x16BCA7C45, 0x16BCA73F0)          = 0 0
fstatat64(0xFFFFFFFFFFFFFFFE, 0x16BCA7C54, 0x16BCA7360)          = 0 0
open("/usr/bin/clear\0", 0x0, 0x0)               = 3 0
fstat64(0x3, 0x16BCA7490, 0x0)           = 0 0
open("/tmp/test\0", 0x401, 0x0)          = 4 0
fstat64(0x4, 0x16BCA75C0, 0x0)           = 0 0
sysctl([CTL_HW, 7, 0, 0, 0, 0] (2), 0x207EC4068, 0x16BCA7280, 0x0, 0x0)          = 0 0
lseek(0x3, 0x0, 0x4)             = -1 Err#6
ftruncate(0x4, 0x1D770, 0x0)             = 0 0
close(0x4)               = 0 0
close(0x3)               = 0 0
lseek(0x0, 0x0, 0x1)             = 146611 0
lseek(0x0, 0x0, 0x1)             = 146611 0
lseek(0x0, 0x23CB3, 0x0)                 = 146611 0
close_nocancel(0x0)              = 0 0
close_nocancel(0x1)              = 0 0
close_nocancel(0x2)              = 0 0

-- 
Cameron Katri
Email: me <at> cameronkatri.com
PGP Fingerprint: 7D3B36CEA40FCC2181FB6DCDBAFFD97826540F1C
[signature.asc (application/pgp-signature, inline)]

Information forwarded to bug-coreutils <at> gnu.org:
bug#51857; Package coreutils. (Mon, 15 Nov 2021 23:22:01 GMT) Full text and rfc822 format available.

Message #29 received at 51857 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Sudhip Nashi <sudhipnashi <at> icloud.com>, Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 51857 <at> debbugs.gnu.org, Cameron Katri <me <at> cameronkatri.com>
Subject: Re: bug#51857: cross-filesystem copying broken on macOS with
 coreutils >= 9.0
Date: Mon, 15 Nov 2021 23:20:08 +0000
[Message part 1 (text/plain, inline)]
On 15/11/2021 18:37, Sudhip Nashi via GNU coreutils Bug Reports wrote:
> 
>> On Nov 15, 2021, at 11:33, Paul Eggert <eggert <at> cs.ucla.edu> wrote:
>>
>> Is the source file on a ZFS file system by any chance? See my lseek comment below.
>>
>>> On 11/15/21 07:48, Cameron Katri via GNU coreutils Bug Reports wrote:
>>>
>>> stat64("/tmp/test\0", 0x16DDC36C0, 0x0)         = 0 0
>>> fstatat64(0xFFFFFFFFFFFFFFFE, 0x16DDC3C21, 0x16DDC2BA0)         = 0 0
>>> fstatat64(0xFFFFFFFFFFFFFFFE, 0x16DDC3C30, 0x16DDC2B10)         = 0 0
>>> open("/usr/bin/clear\0", 0x0, 0x0)         = 3 0
>>> fstat64(0x3, 0x16DDC2C30, 0x0)         = 0 0
>>> open("/tmp/test\0", 0x401, 0x0)         = 4 0
>>> fstat64(0x4, 0x16DDC2CC0, 0x0)         = 0 0
>>> fstat64(0x4, 0x16DDC2D50, 0x0)         = 0 0
>>> fcntl(0x3, 0x32, 0x16DDC3200)         = 0 0
>>> fcntl(0x4, 0x32, 0x16DDC2E00)         = 0 0
>>> unlink("/private/tmp/test\0", 0x0, 0x0)         = 0 0
>>
>> What's causing this use of "/private/tmp"? I don't see that in the GNU cp source code. Can you put a breakpoint on clonefileat and see what's calling it and what its arguments are?
>>
>>> clonefileat(0xFFFFFFFFFFFFFFFE, 0x16DDC3200, 0xFFFFFFFFFFFFFFFE)         = -1 Err#18
>>> open("/private/tmp/test\0", 0x601, 0x81ED)         = 5 0
>>> close(0x5)         = 0 0
>>> open("/private/tmp/test\0", 0x2, 0x0)         = 5 0
>>> dup2(0x5, 0x4, 0x0)         = 4 0
>>> close(0x5)         = 0 0
>>> fchmod(0x4, 0x81ED, 0x0)         = 0 0
>>> fchown(0x4, 0x0, 0x0)         = 0 0
>>> futimes(0x4, 0x16DDC2DE0, 0x0)         = 0 0
>>> sysctl([CTL_HW, 7, 0, 0, 0, 0] (2), 0x207EC4068, 0x16DDC2A30, 0x0, 0x0)         = 0 0
>>> lseek(0x3, 0x0, 0x4)         = -1 Err#6
>>
>> That lseek call looks like OpenZFS bug 11900 <https://github.com/openzfs/zfs/issues/11900>. If you're using ZFS, the bug really should be fixed in your ZFS implementation as it can affect programs other than coreutils and there's no easy workaround (other than to disable efficient copying). Is this something you can look into, or ask someone with macOS and/or ZFS expertise to look into? For more, see <https://bugs.gnu.org/51433>.
>>
>>> ftruncate(0x4, 0x1D770, 0x0)         = 0 0
>>> close(0x4)         = 0 0
>>> close(0x3)         = 0 0
> 
> Turns out lseek is broken (or at least works differently) on macOS as well (https://lists.gnu.org/archive/html/bug-gnulib/2018-09/msg00054.html). Funny coincidence! I’ll take a better look later this week if I can and try to see what the exact problem is.

I saw on other report of failure on macOS.
I think we should disable the SEEK_DATA optimization there for now.
The attached does that.

I'll apply that later.
I also have access to a macOS system, so I'll also test out
if there are ways to use SEEK_DATA there.

thanks,
Pádraig
[seek-data-apple-skip.patch (text/x-patch, attachment)]

Information forwarded to bug-coreutils <at> gnu.org:
bug#51857; Package coreutils. (Mon, 15 Nov 2021 23:31:02 GMT) Full text and rfc822 format available.

Message #32 received at 51857 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Pádraig Brady <P <at> draigBrady.com>,
 Sudhip Nashi <sudhipnashi <at> icloud.com>
Cc: 51857 <at> debbugs.gnu.org, Cameron Katri <me <at> cameronkatri.com>
Subject: Re: bug#51857: cross-filesystem copying broken on macOS with
 coreutils >= 9.0
Date: Mon, 15 Nov 2021 15:30:06 -0800
On 11/15/21 15:20, Pádraig Brady wrote:
> I also have access to a macOS system, so I'll also test out
> if there are ways to use SEEK_DATA there.

Could you try the latest savannah Git instead? I installed something 
into Gnulib that I hope lets coreutils use SEEK_DATA on macOS as before. 
The Gnulib workaround operates on macOS only, and requires an 
lseek+SEEK_HOLE and an lseek+SEEK_DATA where Linux, FreeBSD etc. need 
only lseek+SEEK_DATA, but that's good enough I would think to copy holes 
efficiently on macOS.

I plan to send another email about this shortly.




Information forwarded to bug-coreutils <at> gnu.org:
bug#51857; Package coreutils. (Mon, 15 Nov 2021 23:42:01 GMT) Full text and rfc822 format available.

Message #35 received at 51857 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Sudhip Nashi <sudhipnashi <at> icloud.com>
Cc: 51857 <at> debbugs.gnu.org, Gnulib bugs <bug-gnulib <at> gnu.org>,
 Pádraig Brady <P <at> draigbrady.com>,
 Cameron Katri <me <at> cameronkatri.com>
Subject: Re: bug#51857: cross-filesystem copying broken on macOS with
 coreutils >= 9.0
Date: Mon, 15 Nov 2021 15:41:00 -0800
[Message part 1 (text/plain, inline)]
On 11/15/21 10:37, Sudhip Nashi wrote:
> Turns out lseek is broken (or at least works differently) on macOS as well (https://lists.gnu.org/archive/html/bug-gnulib/2018-09/msg00054.html). Funny coincidence! I’ll take a better look later this week if I can and try to see what the exact problem is.

Thanks, I think I see the problem now.

Eventually macOS will likely get fixed to work around this 
lseek+SEEK_DATA incompatibility (as FreeBSD, Solaris, etc. all do things 
the Linux way and that's what I think will appear in the next POSIX), 
but in the meantime I attempted to work around the portability issue by 
installing the attached patch into Gnulib, and by syncing coreutils to 
the latest Gnulib.

I don't use macOS so have not tested this. Please give it a try, either 
by building from bleeding-edge coreutils on Savannah, or by building 
from the tarball temporarily here:

https://www.cs.ucla.edu/~eggert/coreutils-9.0.26-0f4d9.tar.gz

Thanks.
[0001-lseek-port-around-macOS-SEEK_DATA-glitch.patch (text/x-patch, attachment)]

Information forwarded to bug-coreutils <at> gnu.org:
bug#51857; Package coreutils. (Mon, 15 Nov 2021 23:47:02 GMT) Full text and rfc822 format available.

Message #38 received at 51857 <at> debbugs.gnu.org (full text, mbox):

From: Sudhip Nashi <sudhipnashi <at> icloud.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 51857 <at> debbugs.gnu.org, Gnulib bugs <bug-gnulib <at> gnu.org>,
 Pádraig Brady <P <at> draigbrady.com>,
 Cameron Katri <me <at> cameronkatri.com>
Subject: Re: bug#51857: cross-filesystem copying broken on macOS with
 coreutils >= 9.0
Date: Mon, 15 Nov 2021 17:46:14 -0600
Awesome, thanks for the quick response! I’ll give this a go and let you know the results.

> On Nov 15, 2021, at 5:41 PM, Paul Eggert <eggert <at> cs.ucla.edu> wrote:
> 
> On 11/15/21 10:37, Sudhip Nashi wrote:
>> Turns out lseek is broken (or at least works differently) on macOS as well (https://lists.gnu.org/archive/html/bug-gnulib/2018-09/msg00054.html). Funny coincidence! I’ll take a better look later this week if I can and try to see what the exact problem is.
> 
> Thanks, I think I see the problem now.
> 
> Eventually macOS will likely get fixed to work around this lseek+SEEK_DATA incompatibility (as FreeBSD, Solaris, etc. all do things the Linux way and that's what I think will appear in the next POSIX), but in the meantime I attempted to work around the portability issue by installing the attached patch into Gnulib, and by syncing coreutils to the latest Gnulib.
> 
> I don't use macOS so have not tested this. Please give it a try, either by building from bleeding-edge coreutils on Savannah, or by building from the tarball temporarily here:
> 
> https://www.cs.ucla.edu/~eggert/coreutils-9.0.26-0f4d9.tar.gz
> 
> Thanks.<0001-lseek-port-around-macOS-SEEK_DATA-glitch.patch>





Information forwarded to bug-coreutils <at> gnu.org:
bug#51857; Package coreutils. (Tue, 16 Nov 2021 00:03:02 GMT) Full text and rfc822 format available.

Message #41 received at 51857 <at> debbugs.gnu.org (full text, mbox):

From: Sudhip Nashi <sudhipnashi <at> icloud.com>
To: Sudhip Nashi <sudhipnashi <at> icloud.com>
Cc: 51857 <at> debbugs.gnu.org, Paul Eggert <eggert <at> cs.ucla.edu>,
 Gnulib bugs <bug-gnulib <at> gnu.org>, Cameron Katri <me <at> cameronkatri.com>,
 Pádraig Brady <P <at> draigbrady.com>
Subject: Re: bug#51857: cross-filesystem copying broken on macOS with
 coreutils >= 9.0
Date: Mon, 15 Nov 2021 18:02:10 -0600
It doesn’t seem like this patch works. I compiled from the tarball you provided, but the destination file is still filled with NULL chars.
> On Nov 15, 2021, at 5:46 PM, Sudhip Nashi via GNU coreutils Bug Reports <bug-coreutils <at> gnu.org> wrote:
> 
> Awesome, thanks for the quick response! I’ll give this a go and let you know the results.
> 
>> On Nov 15, 2021, at 5:41 PM, Paul Eggert <eggert <at> cs.ucla.edu> wrote:
>> 
>> On 11/15/21 10:37, Sudhip Nashi wrote:
>>> Turns out lseek is broken (or at least works differently) on macOS as well (https://lists.gnu.org/archive/html/bug-gnulib/2018-09/msg00054.html). Funny coincidence! I’ll take a better look later this week if I can and try to see what the exact problem is.
>> 
>> Thanks, I think I see the problem now.
>> 
>> Eventually macOS will likely get fixed to work around this lseek+SEEK_DATA incompatibility (as FreeBSD, Solaris, etc. all do things the Linux way and that's what I think will appear in the next POSIX), but in the meantime I attempted to work around the portability issue by installing the attached patch into Gnulib, and by syncing coreutils to the latest Gnulib.
>> 
>> I don't use macOS so have not tested this. Please give it a try, either by building from bleeding-edge coreutils on Savannah, or by building from the tarball temporarily here:
>> 
>> https://www.cs.ucla.edu/~eggert/coreutils-9.0.26-0f4d9.tar.gz
>> 
>> Thanks.<0001-lseek-port-around-macOS-SEEK_DATA-glitch.patch>
> 
> 
> 
> 





Information forwarded to bug-coreutils <at> gnu.org:
bug#51857; Package coreutils. (Tue, 16 Nov 2021 02:11:01 GMT) Full text and rfc822 format available.

Message #44 received at 51857 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Sudhip Nashi <sudhipnashi <at> icloud.com>
Cc: 51857 <at> debbugs.gnu.org, Gnulib bugs <bug-gnulib <at> gnu.org>,
 Cameron Katri <me <at> cameronkatri.com>,
 Pádraig Brady <P <at> draigbrady.com>
Subject: Re: bug#51857: cross-filesystem copying broken on macOS with
 coreutils >= 9.0
Date: Mon, 15 Nov 2021 18:10:10 -0800
On 11/15/21 16:02, Sudhip Nashi wrote:
> It doesn’t seem like this patch works. I compiled from the tarball you provided, but the destination file is still filled with NULL chars.

Too bad. Can you do a dtruss or at least a ktrace/kdump for the failing 
program? I'd like to see what lseek is doing with SEEK_HOLE and 
SEEK_DATA. Thanks.




Information forwarded to bug-coreutils <at> gnu.org:
bug#51857; Package coreutils. (Tue, 16 Nov 2021 03:16:02 GMT) Full text and rfc822 format available.

Message #47 received at 51857 <at> debbugs.gnu.org (full text, mbox):

From: Sudhip Nashi <sudhipnashi <at> icloud.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 51857 <at> debbugs.gnu.org, Gnulib bugs <bug-gnulib <at> gnu.org>,
 Cameron Katri <me <at> cameronkatri.com>,
 Pádraig Brady <P <at> draigbrady.com>
Subject: Re: bug#51857: cross-filesystem copying broken on macOS with
 coreutils >= 9.0
Date: Mon, 15 Nov 2021 21:14:44 -0600
Sure, here’s the dtruss output:

SYSCALL(args) 		 = return
access("/AppleInternal/XBS/.isChrooted\0", 0x0, 0x0)		 = -1 Err#2
bsdthread_register(0x1AEC802C8, 0x1AEC802BC, 0x4000)		 = 1073742303 0
shm_open(0x1AEB48F55, 0x0, 0xDD8000)		 = 3 0
fstat64(0x3, 0x16F0260D0, 0x0)		 = 0 0
mmap(0x0, 0x4000, 0x1, 0x40001, 0x3, 0x0)		 = 0x100F18000 0
close(0x3)		 = 0 0
ioctl(0x2, 0x4004667A, 0x16F02617C)		 = -1 Err#25
ioctl(0x2, 0x40487413, 0x16F026180)		 = -1 Err#25
mprotect(0x100F24000, 0x4000, 0x0)		 = 0 0
mprotect(0x100F30000, 0x4000, 0x0)		 = 0 0
mprotect(0x100F34000, 0x4000, 0x0)		 = 0 0
mprotect(0x100F40000, 0x4000, 0x0)		 = 0 0
mprotect(0x100F44000, 0x4000, 0x0)		 = 0 0
mprotect(0x100F50000, 0x4000, 0x0)		 = 0 0
mprotect(0x100F1C000, 0x90, 0x1)		 = 0 0
mprotect(0x100F1C000, 0x90, 0x3)		 = 0 0
mprotect(0x100F1C000, 0x90, 0x1)		 = 0 0
mprotect(0x100F54000, 0x4000, 0x1)		 = 0 0
mprotect(0x100F58000, 0x90, 0x1)		 = 0 0
mprotect(0x100F58000, 0x90, 0x3)		 = 0 0
mprotect(0x100F58000, 0x90, 0x1)		 = 0 0
mprotect(0x100F1C000, 0x90, 0x3)		 = 0 0
mprotect(0x100F1C000, 0x90, 0x1)		 = 0 0
mprotect(0x100F54000, 0x4000, 0x3)		 = 0 0
mprotect(0x100F54000, 0x4000, 0x1)		 = 0 0
objc_bp_assist_cfg_np(0x1AEB103C0, 0x8000000000201048, 0x0)		 = -1 Err#5
issetugid(0x0, 0x0, 0x0)		 = 0 0
getentropy(0x16F025F98, 0x20, 0x0)		 = 0 0
getentropy(0x16F025FE8, 0x40, 0x0)		 = 0 0
getpid(0x0, 0x0, 0x0)		 = 45930 0
stat64("/AppleInternal\0", 0x16F0266E0, 0x0)		 = -1 Err#2
csops_audittoken(0xB36A, 0x7, 0x16F026210)		 = 0 0
proc_info(0x2, 0xB36A, 0xD)		 = 64 0
csops_audittoken(0xB36A, 0x7, 0x16F0262D0)		 = 0 0
sysctlbyname(kern.osvariant_status, 0x15, 0x16F026748, 0x16F026740, 0x0)		 = 0 0
csops(0xB36A, 0x0, 0x16F02676C)		 = 0 0
mprotect(0x100E10000, 0x100000, 0x1)		 = 0 0
open_nocancel("/usr/share/locale/en_US.UTF-8/LC_COLLATE\0", 0x0, 0x0)		 = 3 0
fcntl_nocancel(0x3, 0x3, 0x0)		 = 0 0
getrlimit(0x1008, 0x16F027118, 0x0)		 = 0 0
fstat64(0x3, 0x16F027090, 0x0)		 = 0 0
read_nocancel(0x3, "1.1A\n\0", 0x1000)		 = 2086 0
close_nocancel(0x3)		 = 0 0
open_nocancel("/usr/share/locale/en_US.UTF-8/LC_CTYPE\0", 0x0, 0x0)		 = 3 0
fcntl_nocancel(0x3, 0x3, 0x0)		 = 0 0
fstat64(0x3, 0x16F0271C0, 0x0)		 = 0 0
fstat64(0x3, 0x16F026FB0, 0x0)		 = 0 0
lseek(0x3, 0x0, 0x1)		 = 0 0
lseek(0x3, 0x0, 0x0)		 = 0 0
read_nocancel(0x3, "RuneMagAUTF-8\0", 0x1000)		 = 4096 0
read_nocancel(0x3, "\0", 0x1000)		 = 4096 0
read_nocancel(0x3, "\0", 0x1000)		 = 4096 0
read_nocancel(0x3, "\0", 0x1000)		 = 4096 0
read_nocancel(0x3, "\0", 0x1000)		 = 4096 0
read_nocancel(0x3, "\0", 0x1000)		 = 4096 0
read_nocancel(0x3, "\0", 0x1000)		 = 4096 0
read_nocancel(0x3, "@\004\211\0", 0xF5D0)		 = 62928 0
close_nocancel(0x3)		 = 0 0
open_nocancel("/usr/share/locale/en_US.UTF-8/LC_MONETARY\0", 0x0, 0x0)		 = 3 0
fstat64(0x3, 0x16F0271E0, 0x0)		 = 0 0
read_nocancel(0x3, "USD \n$\n.\n,\n3;3\n\n-\n2\n2\n1\n0\n1\n0\n1\n1\n(\0", 0x22)		 = 34 0
close_nocancel(0x3)		 = 0 0
open_nocancel("/usr/share/locale/en_US.UTF-8/LC_NUMERIC\0", 0x0, 0x0)		 = 3 0
fstat64(0x3, 0x16F0271E0, 0x0)		 = 0 0
read_nocancel(0x3, ".\n,\n3;3\n@$\b\0", 0x8)		 = 8 0
close_nocancel(0x3)		 = 0 0
open_nocancel("/usr/share/locale/en_US.UTF-8/LC_TIME\0", 0x0, 0x0)		 = 3 0
fstat64(0x3, 0x16F0271F0, 0x0)		 = 0 0
read_nocancel(0x3, "Jan\nFeb\nMar\nApr\nMay\nJun\nJul\nAug\nSep\nOct\nNov\nDec\nJanuary\nFebruary\nMarch\nApril\nMay\nJune\nJuly\nAugust\nSeptember\nOctober\nNovember\nDecember\nSun\nMon\nTue\nWed\nThu\nFri\nSat\nSunday\nMonday\nTuesday\nWednesday\nThursday\nFriday\nSaturday\n%H:%M:%S\n%m/%d/%Y\n%a %b %e %X %Y\nAM\nP", 0x179)		 = 377 0
close_nocancel(0x3)		 = 0 0
open_nocancel("/usr/share/locale/en_US.UTF-8/LC_MESSAGES/LC_MESSAGES\0", 0x0, 0x0)		 = 3 0
fstat64(0x3, 0x16F0271F0, 0x0)		 = 0 0
read_nocancel(0x3, "^[yYsS].*\n^[nN].*\n(\0", 0x12)		 = 18 0
close_nocancel(0x3)		 = 0 0
geteuid(0x0, 0x0, 0x0)		 = 0 0
stat64("/tmp/filetest\0", 0x16F0276C0, 0x0)		 = -1 Err#2
fstatat64(0xFFFFFFFFFFFFFFFE, 0x16F027BDD, 0x16F027390)		 = 0 0
fstatat64(0xFFFFFFFFFFFFFFFE, 0x16F027C0B, 0x16F027300)		 = -1 Err#2
open("/System/Library/dyld/dyld_shared_cache_arm64e\0", 0x0, 0x0)		 = 3 0
fstat64(0x3, 0x16F027430, 0x0)		 = 0 0
open("/tmp/filetest\0", 0xA01, 0x1ED)		 = 4 0
fstat64(0x4, 0x16F027560, 0x0)		 = 0 0
sysctl([CTL_HW, 7, 0, 0, 0, 0] (2), 0x207EC4068, 0x16F027220, 0x0, 0x0)		 = 0 0
lseek(0x3, 0x0, 0x3)		 = -1 Err#6
ftruncate(0x4, 0x59C68000, 0x0)		 = 0 0
close(0x4)		 = 0 0
close(0x3)		 = 0 0
lseek(0x0, 0x0, 0x1)		 = 21373 0
lseek(0x0, 0x0, 0x1)		 = 21373 0
lseek(0x0, 0x537D, 0x0)		 = 21373 0
close_nocancel(0x0)		 = 0 0
close_nocancel(0x1)		 = 0 0
close_nocancel(0x2)		 = 0 0


> On Nov 15, 2021, at 8:10 PM, Paul Eggert <eggert <at> cs.ucla.edu> wrote:
> 
> On 11/15/21 16:02, Sudhip Nashi wrote:
>> It doesn’t seem like this patch works. I compiled from the tarball you provided, but the destination file is still filled with NULL chars.
> 
> Too bad. Can you do a dtruss or at least a ktrace/kdump for the failing program? I'd like to see what lseek is doing with SEEK_HOLE and SEEK_DATA. Thanks.





Information forwarded to bug-coreutils <at> gnu.org:
bug#51857; Package coreutils. (Tue, 16 Nov 2021 07:32:02 GMT) Full text and rfc822 format available.

Message #50 received at 51857 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Sudhip Nashi <sudhipnashi <at> icloud.com>
Cc: 51857 <at> debbugs.gnu.org, Gnulib bugs <bug-gnulib <at> gnu.org>,
 Cameron Katri <me <at> cameronkatri.com>,
 Pádraig Brady <P <at> draigbrady.com>
Subject: Re: bug#51857: cross-filesystem copying broken on macOS with
 coreutils >= 9.0
Date: Mon, 15 Nov 2021 23:31:11 -0800
[Message part 1 (text/plain, inline)]
On 11/15/21 19:14, Sudhip Nashi wrote:
> lseek(0x3, 0x0, 0x3)		 = -1 Err#6

Oh my, it appears lseek (fd, 0, SEEK_HOLE) is failing with errno == 
ENXIO when the file has no holes, even though the Darwin man page 
clearly states that lseek should return the file size in that case (see 
<https://github.com/apple/darwin-xnu/blob/main/bsd/man/man2/lseek.2>). 
So, not only does macOS lseek disagree with all other implementations, 
it even disagrees with the Darwin documentation.

To work around this macOS problem I installed the attached further patch 
into Gnulib and propagated it into coreutils. Please try the latest 
coreutils version on Savannah, or you can simply run configure+make from 
the tarball that is temporarily at:

https://web.cs.ucla.edu/~eggert/coreutils-9.0.28-6d0f0.tar.gz
[0001-lseek-port-around-macOS-SEEK_HOLE-glitch.patch (text/x-patch, attachment)]

Information forwarded to bug-coreutils <at> gnu.org:
bug#51857; Package coreutils. (Tue, 16 Nov 2021 07:32:02 GMT) Full text and rfc822 format available.

Message #53 received at 51857 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Sudhip Nashi <sudhipnashi <at> icloud.com>
Cc: 51857 <at> debbugs.gnu.org, Gnulib bugs <bug-gnulib <at> gnu.org>,
 Cameron Katri <me <at> cameronkatri.com>,
 Pádraig Brady <P <at> draigbrady.com>
Subject: Re: bug#51857: cross-filesystem copying broken on macOS with
 coreutils >= 9.0
Date: Mon, 15 Nov 2021 23:31:12 -0800
[Message part 1 (text/plain, inline)]
On 11/15/21 19:14, Sudhip Nashi wrote:
> lseek(0x3, 0x0, 0x3)		 = -1 Err#6

Oh my, it appears lseek (fd, 0, SEEK_HOLE) is failing with errno == 
ENXIO when the file has no holes, even though the Darwin man page 
clearly states that lseek should return the file size in that case (see 
<https://github.com/apple/darwin-xnu/blob/main/bsd/man/man2/lseek.2>). 
So, not only does macOS lseek disagree with all other implementations, 
it even disagrees with the Darwin documentation.

To work around this macOS problem I installed the attached further patch 
into Gnulib and propagated it into coreutils. Please try the latest 
coreutils version on Savannah, or you can simply run configure+make from 
the tarball that is temporarily at:

https://web.cs.ucla.edu/~eggert/coreutils-9.0.28-6d0f0.tar.gz
[0001-lseek-port-around-macOS-SEEK_HOLE-glitch.patch (text/x-patch, attachment)]

Information forwarded to bug-coreutils <at> gnu.org:
bug#51857; Package coreutils. (Tue, 16 Nov 2021 19:16:02 GMT) Full text and rfc822 format available.

Message #56 received at 51857 <at> debbugs.gnu.org (full text, mbox):

From: Sudhip Nashi <sudhipnashi <at> icloud.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 51857 <at> debbugs.gnu.org, Gnulib bugs <bug-gnulib <at> gnu.org>,
 Cameron Katri <me <at> cameronkatri.com>,
 Pádraig Brady <P <at> draigbrady.com>
Subject: Re: bug#51857: cross-filesystem copying broken on macOS with
 coreutils >= 9.0
Date: Tue, 16 Nov 2021 10:40:31 -0600
> So, not only does macOS lseek disagree with all other implementations, it even disagrees with the Darwin documentation.
Oh wow, that’s convoluted.

> To work around this macOS problem I installed the attached further patch into Gnulib and propagated it into coreutils. Please try the latest coreutils version on Savannah, or you can simply run configure+make from the tarball that is temporarily at:
> 
> https://web.cs.ucla.edu/~eggert/coreutils-9.0.28-6d0f0.tar.gz
> <0001-lseek-port-around-macOS-SEEK_HOLE-glitch.patch>
It looks like this patch fixes the problem, and a quick checksum verifies this.



Reply sent to Paul Eggert <eggert <at> cs.ucla.edu>:
You have taken responsibility. (Wed, 17 Nov 2021 00:44:02 GMT) Full text and rfc822 format available.

Notification sent to Sudhip Nashi <sudhipnashi <at> icloud.com>:
bug acknowledged by developer. (Wed, 17 Nov 2021 00:44:02 GMT) Full text and rfc822 format available.

Message #61 received at 51857-done <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Sudhip Nashi <sudhipnashi <at> icloud.com>
Cc: Pádraig Brady <P <at> draigbrady.com>,
 51857-done <at> debbugs.gnu.org, Gnulib bugs <bug-gnulib <at> gnu.org>,
 Cameron Katri <me <at> cameronkatri.com>
Subject: Re: bug#51857: cross-filesystem copying broken on macOS with
 coreutils >= 9.0
Date: Tue, 16 Nov 2021 16:43:16 -0800
On 11/16/21 08:40, Sudhip Nashi via GNU coreutils Bug Reports wrote:

> It looks like this patch fixes the problem, and a quick checksum verifies this.

Thanks for checking. Closing the Coreutils bug report.




bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Wed, 15 Dec 2021 12:24:05 GMT) Full text and rfc822 format available.

This bug report was last modified 2 years and 105 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.