GNU bug report logs - #17827
“Too many open files” error while substituter is running

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 17827 in the body.
You can then email your comments to 17827 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: bug-guix <at> gnu.org
Subject: “Too many open files” error
 while substituter is running
Date: Sat, 21 Jun 2014 16:24:05 +0200

While running the substituter in a disk/VM image built with Guix (on the
metal or on QEMU), I quickly stumble upon EMFILE.  It goes like this:

  1. Run ‘guix build emacs -n’ to warm up /var/guix/substitute-binary;
     this (currently) uses many threads and everything, but doesn’t
     cause any problems.

  2. Run ‘guix build emacs’.  This starts downloading substitutes, via
     ‘guix substitute-binary --substitute’.  It works fine, except that
     running something in parallel like ‘guix package --help’ quickly
     fails with EMFILE (“Too many open files”.)

Looking at /proc/sys/fs/file-nr suggests we’re still way below the
threshold at that point.  Increasing file-max doesn’t make a difference.

Looking at the strace output of the failing ‘guix package --help’ shows
that it does lots of open/close calls, but remains at 10 open files.
Eventually, one of its close(2) calls (!) returns EMFILE, and all the
subsequent open(2) calls fail similarly.

The problem seems to manifest more easily when there’s a lot of RAM,
which made me thing of a Scheme file port leak.

Ludo’.

Message #10 received at 17827-done <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: 17827-done <at> debbugs.gnu.org
Subject: Re: bug#17827: “Too many open files” error while substituter is running
Date: Sat, 21 Jun 2014 17:37:26 +0200

The problem was that the unionfs process for the root file system would
run out of file descriptors (the default limit is 1024.)

Fixed in commit 9b9148d.

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.