GNU bug report logs - #25029
shuf generates “Bad file descriptor” error on nfs but only when run as a background process

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 25029 in the body.
You can then email your comments to 25029 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Alex Ryan <alexander.j.ryan <at> gmail.com>
To: bug-coreutils <at> gnu.org
Subject: shuf generates “Bad file descriptor” error on nfs but only when run as a background process
Date: Fri, 25 Nov 2016 14:24:22 -0800

[Message part 1 (text/plain, inline)]

I am not certain about this but I *think* this might be a bug in shuf.

full description of how to reproduce is here
http://stackoverflow.com/questions/40730239/shuf-generates-bad-file-descriptor-error-on-nfs-but-only-when-run-as-a-backgro


-- 
Alex Ryan
https://www.linkedin.com/in/alexryan

[Message part 2 (text/html, inline)]

Message #8 received at 25029 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Alex Ryan <alexander.j.ryan <at> gmail.com>, 25029 <at> debbugs.gnu.org
Subject: Re: bug#25029: shuf generates “Bad file descriptor” error on nfs but only when run as a background process
Date: Fri, 25 Nov 2016 15:20:23 -0800

I see no real evidence that this is a bug in 'shuf'. It looks like a bug in the 
program that calls 'shuf'. You can verify this by using 'strace shuf' instead of 
plain 'shuf'.

Message #11 received at 25029 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Alex Ryan <alexander.j.ryan <at> gmail.com>, 25029 <at> debbugs.gnu.org
Subject: Re: bug#25029: shuf generates “Bad file descriptor” error on nfs but only when run as a background process
Date: Fri, 25 Nov 2016 23:59:13 +0000

On 25/11/16 22:24, Alex Ryan wrote:
> I am not certain about this but I *think* this might be a bug in shuf.
> 
> full description of how to reproduce is here
> http://stackoverflow.com/questions/40730239/shuf-generates-bad-file-descriptor-error-on-nfs-but-only-when-run-as-a-backgro

The minimum reproducer is:

  $ shuf 0>&-
  shuf: read error: Bad file descriptor

The reason is that the specified file is reopened to stdin,
which is a general technique used by single file filters.
I.E. this isn't specific to shuf:

  $ tac 0>&-
  tac: standard input: read error: Bad file descriptor

You can avoid the issue by redirecting stdin from the file like:

  $ shuf < your_file

cheers,
Pádraig.

Message #14 received at 25029 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Alex Ryan <alexander.j.ryan <at> gmail.com>, 25029 <at> debbugs.gnu.org
Subject: Re: bug#25029: shuf generates “Bad file descriptor” error on nfs but only when run as a background process
Date: Sat, 26 Nov 2016 00:06:49 +0000

On 25/11/16 23:59, Pádraig Brady wrote:
> On 25/11/16 22:24, Alex Ryan wrote:
>> I am not certain about this but I *think* this might be a bug in shuf.
>>
>> full description of how to reproduce is here
>> http://stackoverflow.com/questions/40730239/shuf-generates-bad-file-descriptor-error-on-nfs-but-only-when-run-as-a-backgro
> 
> The minimum reproducer is:
> 
>   $ shuf 0>&-
>   shuf: read error: Bad file descriptor
> 
> The reason is that the specified file is reopened to stdin,
> which is a general technique used by single file filters.
> I.E. this isn't specific to shuf:
> 
>   $ tac 0>&-
>   tac: standard input: read error: Bad file descriptor
> 
> You can avoid the issue by redirecting stdin from the file like:
> 
>   $ shuf < your_file

Actually the minimum reproducer is:

  $ shuf /dev/null 0>&-
  shuf: read error: Bad file descriptor

That format shows that tac doesn't have the issue
(and tac is not a single file util anyway):
but uniq does have the issue:

  $ uniq /dev/null 0>&-
  uniq: error reading /dev/null

We might be able to improve this.
Anyway the workaround presented previously should work.

cheers,
Pádraig

Message #17 received at 25029 <at> debbugs.gnu.org (full text, mbox):

From: Alex Ryan <alexander.j.ryan <at> gmail.com>
To: Pádraig Brady <P <at> draigbrady.com>
Cc: 25029 <at> debbugs.gnu.org
Subject: Re: bug#25029: shuf generates “Bad file descriptor” error on nfs but only when run as a background process
Date: Fri, 25 Nov 2016 16:39:24 -0800

[Message part 1 (text/plain, inline)]

That worked beautifully.
Great explanation.
Much appreciated.

On Fri, Nov 25, 2016 at 4:06 PM, Pádraig Brady <P <at> draigbrady.com> wrote:

> On 25/11/16 23:59, Pádraig Brady wrote:
> > On 25/11/16 22:24, Alex Ryan wrote:
> >> I am not certain about this but I *think* this might be a bug in shuf.
> >>
> >> full description of how to reproduce is here
> >> http://stackoverflow.com/questions/40730239/shuf-
> generates-bad-file-descriptor-error-on-nfs-but-only-when-run-as-a-backgro
> >
> > The minimum reproducer is:
> >
> >   $ shuf 0>&-
> >   shuf: read error: Bad file descriptor
> >
> > The reason is that the specified file is reopened to stdin,
> > which is a general technique used by single file filters.
> > I.E. this isn't specific to shuf:
> >
> >   $ tac 0>&-
> >   tac: standard input: read error: Bad file descriptor
> >
> > You can avoid the issue by redirecting stdin from the file like:
> >
> >   $ shuf < your_file
>
> Actually the minimum reproducer is:
>
>   $ shuf /dev/null 0>&-
>   shuf: read error: Bad file descriptor
>
> That format shows that tac doesn't have the issue
> (and tac is not a single file util anyway):
> but uniq does have the issue:
>
>   $ uniq /dev/null 0>&-
>   uniq: error reading /dev/null
>
> We might be able to improve this.
> Anyway the workaround presented previously should work.
>
> cheers,
> Pádraig
>



-- 
Alex Ryan
https://www.linkedin.com/in/alexryan

[Message part 2 (text/html, inline)]

Message #20 received at 25029 <at> debbugs.gnu.org (full text, mbox):

From: Alex Ryan <alexander.j.ryan <at> gmail.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 25029 <at> debbugs.gnu.org
Subject: Re: bug#25029: shuf generates “Bad file descriptor” error on nfs but only when run as a background process
Date: Fri, 25 Nov 2016 16:43:35 -0800

[Message part 1 (text/plain, inline)]

fyi ...
here is the output of strace when the Bad file descriptor error is
generated ...

$nohup ./run_shuffle.sh > /tmp/run_shuffle.log 2>&1 0>&- &

execve("/usr/bin/shuf", ["shuf", "/nfs/digits/datasets/com-aosvapp"...],
[/* 54 vars */]) = 0

brk(0)                                  = 0x10dc000

access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or
directory)

mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x7f9af2d5a000

access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or
directory)

open("tls/x86_64/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file
or directory)

open("tls/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or
directory)

open("x86_64/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or
directory)

open("libc.so.6", O_RDONLY|O_CLOEXEC)   = -1 ENOENT (No such file or
directory)

open("/usr/local/cuda-7.5/lib64/tls/x86_64/libc.so.6", O_RDONLY|O_CLOEXEC)
= -1 ENOENT (No such file or directory)

stat("/usr/local/cuda-7.5/lib64/tls/x86_64", 0x7ffcc850b6c0) = -1 ENOENT
(No such file or directory)

open("/usr/local/cuda-7.5/lib64/tls/libc.so.6", O_RDONLY|O_CLOEXEC) = -1
ENOENT (No such file or directory)

stat("/usr/local/cuda-7.5/lib64/tls", 0x7ffcc850b6c0) = -1 ENOENT (No such
file or directory)

open("/usr/local/cuda-7.5/lib64/x86_64/libc.so.6", O_RDONLY|O_CLOEXEC) = -1
ENOENT (No such file or directory)

stat("/usr/local/cuda-7.5/lib64/x86_64", 0x7ffcc850b6c0) = -1 ENOENT (No
such file or directory)

open("/usr/local/cuda-7.5/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT
(No such file or directory)

stat("/usr/local/cuda-7.5/lib64", {st_mode=S_IFDIR|0755, st_size=4096,
...}) = 0

open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 0

fstat(0, {st_mode=S_IFREG|0644, st_size=78541, ...}) = 0

mmap(NULL, 78541, PROT_READ, MAP_PRIVATE, 0, 0) = 0x7f9af2d46000

close(0)                                = 0

access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or
directory)

open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 0

read(0, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P \2\0\0\0\0\0"...,
832) = 832

fstat(0, {st_mode=S_IFREG|0755, st_size=1840928, ...}) = 0

mmap(NULL, 3949248, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 0, 0) =
0x7f9af2775000

mprotect(0x7f9af292f000, 2097152, PROT_NONE) = 0

mmap(0x7f9af2b2f000, 24576, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 0, 0x1ba000) = 0x7f9af2b2f000

mmap(0x7f9af2b35000, 17088, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f9af2b35000

close(0)                                = 0

mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x7f9af2d45000

mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x7f9af2d43000

arch_prctl(ARCH_SET_FS, 0x7f9af2d43740) = 0

mprotect(0x7f9af2b2f000, 16384, PROT_READ) = 0

mprotect(0x60a000, 4096, PROT_READ)     = 0

mprotect(0x7f9af2d5c000, 4096, PROT_READ) = 0

munmap(0x7f9af2d46000, 78541)           = 0

brk(0)                                  = 0x10dc000

brk(0x10fd000)                          = 0x10fd000

open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 0

fstat(0, {st_mode=S_IFREG|0644, st_size=1607664, ...}) = 0

mmap(NULL, 1607664, PROT_READ, MAP_PRIVATE, 0, 0) = 0x7f9af2bba000

close(0)                                = 0

open("/nfs/digits/datasets/com-aosvapps-distracted-driving3/databases/TrainImagePathsAndLabels_AlpineTest1.csv",
O_RDONLY) = 0

dup3(0, 0, 0)                           = -1 EINVAL (Invalid argument)

dup2(0, 0)                              = 0

close(0)                                = 0

fadvise64(0, 0, 0, POSIX_FADV_SEQUENTIAL) = -1 EBADF (Bad file descriptor)

fstat(0, 0x7ffcc850be10)                = -1 EBADF (Bad file descriptor)

fstat(0, 0x7ffcc850bcd0)                = -1 EBADF (Bad file descriptor)

mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x7f9af2d58000

read(0, 0x10dd030, 8192)                = -1 EBADF (Bad file descriptor)

open("/usr/share/locale/locale.alias", O_RDONLY|O_CLOEXEC) = 0

fstat(0, {st_mode=S_IFREG|0644, st_size=2570, ...}) = 0

mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x7f9af2d57000

read(0, "# Locale name alias data base.\n#"..., 4096) = 2570

read(0, "", 4096)                       = 0

close(0)                                = 0

munmap(0x7f9af2d57000, 4096)            = 0

open("/usr/share/locale/en_US.UTF-8/LC_MESSAGES/coreutils.mo", O_RDONLY) =
-1 ENOENT (No such file or directory)

open("/usr/share/locale/en_US.utf8/LC_MESSAGES/coreutils.mo", O_RDONLY) =
-1 ENOENT (No such file or directory)

open("/usr/share/locale/en_US/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1
ENOENT (No such file or directory)

open("/usr/share/locale/en.UTF-8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1
ENOENT (No such file or directory)

open("/usr/share/locale/en.utf8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1
ENOENT (No such file or directory)

open("/usr/share/locale/en/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT
(No such file or directory)

open("/usr/share/locale-langpack/en_US.UTF-8/LC_MESSAGES/coreutils.mo",
O_RDONLY) = -1 ENOENT (No such file or directory)

open("/usr/share/locale-langpack/en_US.utf8/LC_MESSAGES/coreutils.mo",
O_RDONLY) = -1 ENOENT (No such file or directory)

open("/usr/share/locale-langpack/en_US/LC_MESSAGES/coreutils.mo", O_RDONLY)
= -1 ENOENT (No such file or directory)

open("/usr/share/locale-langpack/en.UTF-8/LC_MESSAGES/coreutils.mo",
O_RDONLY) = -1 ENOENT (No such file or directory)

open("/usr/share/locale-langpack/en.utf8/LC_MESSAGES/coreutils.mo",
O_RDONLY) = -1 ENOENT (No such file or directory)

open("/usr/share/locale-langpack/en/LC_MESSAGES/coreutils.mo", O_RDONLY) =
-1 ENOENT (No such file or directory)

write(2, "shuf: ", 6shuf: )                   = 6

write(2, "read error", 10read error)              = 10

open("/usr/share/locale/en_US.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = -1
ENOENT (No such file or directory)

open("/usr/share/locale/en_US.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1
ENOENT (No such file or directory)

open("/usr/share/locale/en_US/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT
(No such file or directory)

open("/usr/share/locale/en.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = -1
ENOENT (No such file or directory)

open("/usr/share/locale/en.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT
(No such file or directory)

open("/usr/share/locale/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No
such file or directory)

open("/usr/share/locale-langpack/en_US.UTF-8/LC_MESSAGES/libc.mo",
O_RDONLY) = -1 ENOENT (No such file or directory)

open("/usr/share/locale-langpack/en_US.utf8/LC_MESSAGES/libc.mo", O_RDONLY)
= -1 ENOENT (No such file or directory)

open("/usr/share/locale-langpack/en_US/LC_MESSAGES/libc.mo", O_RDONLY) = -1
ENOENT (No such file or directory)

open("/usr/share/locale-langpack/en.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) =
-1 ENOENT (No such file or directory)

open("/usr/share/locale-langpack/en.utf8/LC_MESSAGES/libc.mo", O_RDONLY) =
-1 ENOENT (No such file or directory)

open("/usr/share/locale-langpack/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1
ENOENT (No such file or directory)

write(2, ": Bad file descriptor", 21: Bad file descriptor)   = 21

write(2, "\n", 1

)                       = 1

close(1)                                = 0

close(2)                                = 0

exit_group(1)                           = ?
+++ exited with 1 +++



On Fri, Nov 25, 2016 at 3:20 PM, Paul Eggert <eggert <at> cs.ucla.edu> wrote:

> I see no real evidence that this is a bug in 'shuf'. It looks like a bug
> in the program that calls 'shuf'. You can verify this by using 'strace
> shuf' instead of plain 'shuf'.
>



-- 
Alex Ryan
https://www.linkedin.com/in/alexryan

[Message part 2 (text/html, inline)]

Message #23 received at 25029 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Pádraig Brady <P <at> draigBrady.com>,
 Alex Ryan <alexander.j.ryan <at> gmail.com>, 25029 <at> debbugs.gnu.org
Subject: Re: bug#25029: shuf generates “Bad file descriptor” error on nfs but only when run as a background process
Date: Fri, 25 Nov 2016 18:00:11 -0800

Pádraig Brady wrote:
> Actually the minimum reproducer is:
>
>   $ shuf /dev/null 0>&-
>   shuf: read error: Bad file descriptor

Ouch. Isn't that a bug in glibc freopen? It shouldn't fail with errno==EINVAL 
merely because stdin doesn't have a valid file descriptor. If so, we should fix 
the Gnulib freopen module to work around the bug. If not, I suppose we can fix 
this in freopen-safer.

Message #26 received at 25029 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Pádraig Brady <P <at> draigBrady.com>,
 Alex Ryan <alexander.j.ryan <at> gmail.com>, 25029 <at> debbugs.gnu.org
Subject: Re: bug#25029: shuf generates “Bad file descriptor” error on nfs but only when run as a background process
Date: Fri, 25 Nov 2016 18:30:41 -0800

Paul Eggert wrote:
> Isn't that a bug in glibc freopen?

It turns out to be a known bug; glibc bug#15589 and glibc bug#15701. I have 
added notes there about this:

https://sourceware.org/bugzilla/show_bug.cgi?id=15589
https://sourceware.org/bugzilla/show_bug.cgi?id=15701

Message #29 received at 25029 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>, Alex Ryan <alexander.j.ryan <at> gmail.com>, 
 25029 <at> debbugs.gnu.org
Subject: Re: bug#25029: shuf generates “Bad file descriptor” error on nfs but only when run as a background process
Date: Sat, 26 Nov 2016 02:41:23 +0000

On 26/11/16 02:00, Paul Eggert wrote:
> Pádraig Brady wrote:
>> Actually the minimum reproducer is:
>>
>>   $ shuf /dev/null 0>&-
>>   shuf: read error: Bad file descriptor
> 
> Ouch. Isn't that a bug in glibc freopen? It shouldn't fail with errno==EINVAL 
> merely because stdin doesn't have a valid file descriptor. If so, we should fix 
> the Gnulib freopen module to work around the bug. If not, I suppose we can fix 
> this in freopen-safer.

Yes it's surprising.  I see POSIX states freopen() should only return
EBADF in the case where the pathname specified is NULL.
I.E. when operating on the old descriptor.
But shouldn't freopen() be using a new descriptor here?
Definitely looks like a glibc bug.

cheers,
Pádraig

Message #34 received at 25029-done <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: 25029-done <at> debbugs.gnu.org
Cc: bug-gnulib <at> gnu.org, Pádraig Brady <P <at> draigBrady.com>,
 Alex Ryan <alexander.j.ryan <at> gmail.com>
Subject: Re: bug#25029: shuf generates “Bad file descriptor” error on nfs but only when run as a background process
Date: Sat, 26 Nov 2016 15:49:51 -0800

[Message part 1 (text/plain, inline)]

On 11/25/2016 06:00 PM, Paul Eggert wrote:
> Isn't that a bug in glibc freopen? It shouldn't fail ... merely 
> because stdin doesn't have a valid file descriptor. If so, we should 
> fix the Gnulib freopen module to work around the bug.

I did that, by installing the attached patches into Gnulib (first patch) 
and into Coreutils (2nd and 3rd patches). This fixes the shuf bug for 
me, so closing the bug report. CC'ing to bug-gnulib due to the Gnulib fix.

[freopen.diff (text/x-patch, attachment)]

[0001-build-update-gnulib-submodule-to-latest.patch (text/x-patch, attachment)]

[0002-shuf-test-input-closed-bug.patch (text/x-patch, attachment)]

Message #37 received at 25029 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>, 25029 <at> debbugs.gnu.org
Cc: bug-gnulib <at> gnu.org, Alex Ryan <alexander.j.ryan <at> gmail.com>
Subject: Re: bug#25029: shuf generates “Bad file descriptor” error on nfs but only when run as a background process
Date: Sun, 27 Nov 2016 00:18:23 +0000

On 26/11/16 23:49, Paul Eggert wrote:
> On 11/25/2016 06:00 PM, Paul Eggert wrote:
>> Isn't that a bug in glibc freopen? It shouldn't fail ... merely 
>> because stdin doesn't have a valid file descriptor. If so, we should 
>> fix the Gnulib freopen module to work around the bug.
> 
> I did that, by installing the attached patches into Gnulib (first patch) 
> and into Coreutils (2nd and 3rd patches). This fixes the shuf bug for 
> me, so closing the bug report. CC'ing to bug-gnulib due to the Gnulib fix.

Cool thanks.  I see it does the equivalent of:

  $tool file 0>&- || $tool file </dev/null

Does the freopen module now need to depend on
the dup2 and open modules?  For example using dup2(fd, fd)
to test fd validity fails on Linux 2.6.29

thanks,
Pádraig

Message #40 received at 25029 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Pádraig Brady <P <at> draigBrady.com>, 25029 <at> debbugs.gnu.org
Cc: bug-gnulib <at> gnu.org, Alex Ryan <alexander.j.ryan <at> gmail.com>
Subject: Re: bug#25029: shuf generates “Bad file descriptor” error on nfs but only when run as a background process
Date: Sat, 26 Nov 2016 16:27:26 -0800

On 11/26/2016 04:18 PM, Pádraig Brady wrote:
> Does the freopen module now need to depend on
> the dup2 and open modules?  For example using dup2(fd, fd)
> to test fd validity fails on Linux 2.6.29

I tried to avoid that dependency, by using code that checks errno after 
dup2 fails, so that the workaround should be attempted only on hosts 
with the more-recent freopen bug. I haven't tested it on ancient 
GNU/Linux systems, though.

Message #43 received at 25029 <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: Pádraig Brady <P <at> draigBrady.com>,
 Alex Ryan <alexander.j.ryan <at> gmail.com>, 25029 <at> debbugs.gnu.org
Subject: Re: bug#25029: shuf generates “Bad file descriptor” error on nfs but only when run as a background process
Date: Mon, 28 Nov 2016 10:37:16 -0600

[Message part 1 (text/plain, inline)]

On 11/25/2016 05:59 PM, Pádraig Brady wrote:
> On 25/11/16 22:24, Alex Ryan wrote:
>> I am not certain about this but I *think* this might be a bug in shuf.
>>
>> full description of how to reproduce is here
>> http://stackoverflow.com/questions/40730239/shuf-generates-bad-file-descriptor-error-on-nfs-but-only-when-run-as-a-backgro
> 
> The minimum reproducer is:
> 
>   $ shuf 0>&-
>   shuf: read error: Bad file descriptor
> 
> The reason is that the specified file is reopened to stdin,
> which is a general technique used by single file filters.
> I.E. this isn't specific to shuf:
> 
>   $ tac 0>&-
>   tac: standard input: read error: Bad file descriptor

Technically, POSIX states that ANY attempt to execute a program with fd
0, 1, or 2 closed is undefined behavior, so the bug is yours for not
providing an explicit stdin in the first place.  But I agree that it is
a nice quality-of-implementation thing for coreutils to have
reasonably-sane behavior in the fact of closed fds, even if POSIX does
not require us to.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.