GNU bug report logs - #36443
Canonicalized build directory name in container leads to confusion

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 36443 in the body.
You can then email your comments to 36443 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Robert Vollmert <rob <at> vllmrt.net>
To: bug-guix <at> gnu.org
Subject: guix build mixes build dirs?
Date: Sun, 30 Jun 2019 12:24:16 +0200

So this is pretty bizarre, and I haven’t managed to cut it down
to a smaller example yet, but it seems pretty clear that something
is broken:

$ guix build -K some-package
-> error, referencing /tmp/guix-build-puzzledb-frontend-20190625-git.drv-0
note: keeping build directory `/tmp/guix-build-puzzledb-frontend-20190625-git.drv-0’
$ guix build -K some-package
-> same error, again referencing /tmp/guix-build-puzzledb-frontend-20190625-git.drv-0
note: keeping build directory `/tmp/guix-build-puzzledb-frontend-20190625-git.drv-1’

My concrete packaging setup is a bit more complicated. I’m working with elm, and
faking the build directory as the home directory. The error message mentioned above
is

> The binary data at
> /tmp/guix-build-puzzledb-frontend-20190625-git.drv-0/.elm/0.19.0/package/versions.dat
> is corrupt.

Elm caches build artifacts in the following directories:

    /tmp/guix-build-puzzledb-frontend-20190625-git.drv-0/.elm
    elm-stuff/

while the package definition uses the trivial build system as such:

    (arguments
     `(#:modules ((guix build utils) (build-elm) (json parser) (versions))
       #:builder
         (begin
             …
             (setenv "HOME" (getcwd))
             (setenv "HTTP_PROXY" ".”) ; break http access
             (copy-recursively (assoc-ref %build-inputs "source") "src")
             (with-directory-excursion “src"
               …
               (invoke elm "make" "--output=../all.js" "src/All.elm”)))

The path in the error above comes from $HOME — is there a chance that this gets saved
somewhere? Other parts of the build script appear to work with the -1 directory as
expected.

I’m not at all sure that my package definition is even close to correct, but as far
as I can tell, a mix-up as above should be impossible.

Message #8 received at 36443 <at> debbugs.gnu.org (full text, mbox):

From: Robert Vollmert <rob <at> vllmrt.net>
To: 36443 <at> debbugs.gnu.org
Subject: suspicion
Date: Sun, 30 Jun 2019 14:13:56 +0200

My suspicion right now is that there’s not any real mixup
happening, but that guix-daemon runs the build in a
different environment in the directory

/tmp/derivation-name.drv-0

which gets copied to

/tmp/derivation-name.drv-5

in the root filesystem by `guix build -K`, if directories
-0, -1, …, -4 already exist. In which case I’d suggest doing
something to the messages and/or documentation to make this
clearer.

E.g. instead of saying “keeping build directory <dir>” say
“keeping a copy of the build directory at <dir>”.

Message #11 received at 36443 <at> debbugs.gnu.org (full text, mbox):

From: Mark H Weaver <mhw <at> netris.org>
To: Robert Vollmert <rob <at> vllmrt.net>
Cc: 36443 <at> debbugs.gnu.org
Subject: Re: bug#36443: guix build mixes build dirs?
Date: Sun, 30 Jun 2019 13:18:45 -0400

Robert Vollmert <rob <at> vllmrt.net> writes:

> So this is pretty bizarre, and I haven’t managed to cut it down
> to a smaller example yet, but it seems pretty clear that something
> is broken:
>
> $ guix build -K some-package
> -> error, referencing /tmp/guix-build-puzzledb-frontend-20190625-git.drv-0
> note: keeping build directory `/tmp/guix-build-puzzledb-frontend-20190625-git.drv-0’
> $ guix build -K some-package
> -> same error, again referencing /tmp/guix-build-puzzledb-frontend-20190625-git.drv-0
> note: keeping build directory `/tmp/guix-build-puzzledb-frontend-20190625-git.drv-1’

This is known behavior, and intentional.  The build is done within a
container, and the directory name within the container is canonicalized.

     Regards,
       Mark

Message #14 received at 36443 <at> debbugs.gnu.org (full text, mbox):

From: Robert Vollmert <rob <at> vllmrt.net>
To: Mark H Weaver <mhw <at> netris.org>
Cc: 36443 <at> debbugs.gnu.org
Subject: Re: bug#36443: guix build mixes build dirs?
Date: Sun, 30 Jun 2019 19:22:04 +0200

> On 30. Jun 2019, at 19:18, Mark H Weaver <mhw <at> netris.org> wrote:
> 
> Robert Vollmert <rob <at> vllmrt.net> writes:
> 
>> So this is pretty bizarre, and I haven’t managed to cut it down
>> to a smaller example yet, but it seems pretty clear that something
>> is broken:
>> 
>> $ guix build -K some-package
>> -> error, referencing /tmp/guix-build-puzzledb-frontend-20190625-git.drv-0
>> note: keeping build directory `/tmp/guix-build-puzzledb-frontend-20190625-git.drv-0’
>> $ guix build -K some-package
>> -> same error, again referencing /tmp/guix-build-puzzledb-frontend-20190625-git.drv-0
>> note: keeping build directory `/tmp/guix-build-puzzledb-frontend-20190625-git.drv-1’
> 
> This is known behavior, and intentional.  The build is done within a
> container, and the directory name within the container is canonicalized.

Thanks for clarifying. Do you have an idea how to make this less confusing?

In my follow-up to the bug report I suggested

> E.g. instead of saying “keeping build directory <dir>” say
> “Keeping a copy of the build directory at <dir>”.

But I feel that might be improved upon. “Copied build directory to …”?
What do you think?

Message #17 received at 36443 <at> debbugs.gnu.org (full text, mbox):

From: Mark H Weaver <mhw <at> netris.org>
To: Robert Vollmert <rob <at> vllmrt.net>
Cc: 36443 <at> debbugs.gnu.org
Subject: Re: bug#36443: guix build mixes build dirs?
Date: Sun, 30 Jun 2019 13:30:59 -0400

Mark H Weaver <mhw <at> netris.org> writes:

> Robert Vollmert <rob <at> vllmrt.net> writes:
>
>> So this is pretty bizarre, and I haven’t managed to cut it down
>> to a smaller example yet, but it seems pretty clear that something
>> is broken:
>>
>> $ guix build -K some-package
>> -> error, referencing /tmp/guix-build-puzzledb-frontend-20190625-git.drv-0
>> note: keeping build directory `/tmp/guix-build-puzzledb-frontend-20190625-git.drv-0’
>> $ guix build -K some-package
>> -> same error, again referencing /tmp/guix-build-puzzledb-frontend-20190625-git.drv-0
>> note: keeping build directory `/tmp/guix-build-puzzledb-frontend-20190625-git.drv-1’
>
> This is known behavior, and intentional.  The build is done within a
> container, and the directory name within the container is canonicalized.

FYI, here's the relevant commit that added this behavior:

--8<---------------cut here---------------start------------->8---
commit cb9601029ea164b86bdf997f7160d494c15d344b
Author: Eelco Dolstra <eelco.dolstra <at> logicblox.com>
Date:   Wed Dec 2 14:59:07 2015 +0100

    daemon: Use deterministic $TMPDIR in chroot.
    
    Rather than using $<host-TMPDIR>/nix-build-<drvname>-<number>, the
    temporary directory is now always /tmp/nix-build-<drvname>-0. This
    improves bitwise-exact reproducibility for builds that store $TMPDIR
    in their build output. (Of course, those should still be fixed...)
    
    * nix/libstore/build.cc (DerivationGoal)[tmpDirInSandbox]: New field.
    (DerivationGoal::startBuilder): Initialize 'useChroot' earlier.  Compute
    'tmpDirInSandbox', and use it when populating 'dirsInChroot'.
    * doc/guix.texi (Build Environment Setup): Document it.
    
    Co-authored-by: Ludovic Courtès <ludo <at> gnu.org>
--8<---------------cut here---------------end--------------->8---

       Mark

Message #20 received at 36443 <at> debbugs.gnu.org (full text, mbox):

From: Mark H Weaver <mhw <at> netris.org>
To: Robert Vollmert <rob <at> vllmrt.net>
Cc: 36443 <at> debbugs.gnu.org
Subject: Re: bug#36443: Canonicalized build directory name in container leads
 to confusion (was guix build mixes build dirs?)
Date: Sun, 30 Jun 2019 13:43:26 -0400

retitle 36443 Canonicalized build directory name in container leads to confusion
thanks

Hi Robert,

Robert Vollmert <rob <at> vllmrt.net> writes:
> Thanks for clarifying. Do you have an idea how to make this less confusing?
>
> In my follow-up to the bug report I suggested
>
>> E.g. instead of saying “keeping build directory <dir>” say
>> “Keeping a copy of the build directory at <dir>”.
>
> But I feel that might be improved upon. “Copied build directory to …”?
> What do you think?

The directory is not copied.  It is the same directory, but with a
different name in the build container's namespace.  You can see this by
examining the contents of the directory while the build is in progress.

I agree that it would be good to eliminate this potential source of
confusion, but it's not yet clear to me how to do it.

Any other ideas?

     Regards,
       Mark

Message #25 received at 36443 <at> debbugs.gnu.org (full text, mbox):

From: Robert Vollmert <rob <at> vllmrt.net>
To: Mark H Weaver <mhw <at> netris.org>
Cc: 36443 <at> debbugs.gnu.org
Subject: Re: bug#36443: Canonicalized build directory name in container leads
 to confusion (was guix build mixes build dirs?)
Date: Sun, 30 Jun 2019 19:47:44 +0200

Hi Mark,

> On 30. Jun 2019, at 19:43, Mark H Weaver <mhw <at> netris.org> wrote:
> 
> retitle 36443 Canonicalized build directory name in container leads to confusion
> thanks
> 
> Hi Robert,
> 
> Robert Vollmert <rob <at> vllmrt.net> writes:
>> Thanks for clarifying. Do you have an idea how to make this less confusing?
>> 
>> In my follow-up to the bug report I suggested
>> 
>>> E.g. instead of saying “keeping build directory <dir>” say
>>> “Keeping a copy of the build directory at <dir>”.
>> 
>> But I feel that might be improved upon. “Copied build directory to …”?
>> What do you think?
> 
> The directory is not copied.  It is the same directory, but with a
> different name in the build container's namespace.  You can see this by
> examining the contents of the directory while the build is in progress.
> 
> I agree that it would be good to eliminate this potential source of
> confusion, but it's not yet clear to me how to do it.
> 
> Any other ideas?

I see, thanks for clarifying.

How about dropping the “-0” suffix inside the container? The major part
of my confusion was that the directory “-0” actually existed in /tmp
from a previous failed build; this change might avoid that.

Cheers
Robert

Message #28 received at 36443 <at> debbugs.gnu.org (full text, mbox):

From: Mark H Weaver <mhw <at> netris.org>
To: Robert Vollmert <rob <at> vllmrt.net>
Cc: 36443 <at> debbugs.gnu.org, ludo <at> gnu.org
Subject: Re: bug#36443: Canonicalized build directory name in container leads
 to confusion (was guix build mixes build dirs?)
Date: Sun, 30 Jun 2019 13:59:30 -0400

Hi Robert,

Robert Vollmert <rob <at> vllmrt.net> writes:
> How about dropping the “-0” suffix inside the container? The major part
> of my confusion was that the directory “-0” actually existed in /tmp
> from a previous failed build; this change might avoid that.

Sounds good to me.  I think that would clearly be an improvement.
See below for an (untested but trivial) patch to make this change.

Ludovic, what do you think?  See <https://bugs.gnu.org/36443> for
context.

     Thanks,
       Mark

--8<---------------cut here---------------start------------->8---
diff --git a/nix/libstore/build.cc b/nix/libstore/build.cc
index fe7bf79069..a73f79eed5 100644
--- a/nix/libstore/build.cc
+++ b/nix/libstore/build.cc
@@ -1726,7 +1726,7 @@ void DerivationGoal::startBuilder()
 
     /* In a sandbox, for determinism, always use the same temporary
        directory. */
-    tmpDirInSandbox = useChroot ? canonPath("/tmp", true) + "/guix-build-" + drvName + "-0" : tmpDir;
+    tmpDirInSandbox = useChroot ? canonPath("/tmp", true) + "/guix-build-" + drvName : tmpDir;
 
     /* For convenience, set an environment pointing to the top build
        directory. */
--8<---------------cut here---------------end--------------->8---

Message #31 received at 36443 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Mark H Weaver <mhw <at> netris.org>
Cc: 36443 <at> debbugs.gnu.org, Robert Vollmert <rob <at> vllmrt.net>
Subject: Re: bug#36443: Canonicalized build directory name in container leads
 to confusion (was guix build mixes build dirs?)
Date: Tue, 02 Jul 2019 15:37:58 +0200

Hello,

Mark H Weaver <mhw <at> netris.org> skribis:

> Robert Vollmert <rob <at> vllmrt.net> writes:
>> How about dropping the “-0” suffix inside the container? The major part
>> of my confusion was that the directory “-0” actually existed in /tmp
>> from a previous failed build; this change might avoid that.

Bah, I see.

> Sounds good to me.  I think that would clearly be an improvement.
> See below for an (untested but trivial) patch to make this change.
>
> Ludovic, what do you think?  See <https://bugs.gnu.org/36443> for
> context.

[...]

>      /* In a sandbox, for determinism, always use the same temporary
>         directory. */
> -    tmpDirInSandbox = useChroot ? canonPath("/tmp", true) + "/guix-build-" + drvName + "-0" : tmpDir;
> +    tmpDirInSandbox = useChroot ? canonPath("/tmp", true) + "/guix-build-" + drvName : tmpDir;

The result would be that the temporary directory would always have a
different name inside and outside the container.  Consequently,
debugging along the lines of what the manual suggests (info "(guix)
Debugging Build Failures") would become pretty much impossible.

Thus, I’m in favor of the status quo.

Thoughts?

Ludo’.

Message #34 received at 36443 <at> debbugs.gnu.org (full text, mbox):

From: Robert Vollmert <rob <at> vllmrt.net>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Mark H Weaver <mhw <at> netris.org>, 36443 <at> debbugs.gnu.org
Subject: Re: bug#36443: Canonicalized build directory name in container leads
 to confusion (was guix build mixes build dirs?)
Date: Tue, 2 Jul 2019 15:41:00 +0200

> On 2. Jul 2019, at 15:37, Ludovic Courtès <ludo <at> gnu.org> wrote:
> 
>>     /* In a sandbox, for determinism, always use the same temporary
>>        directory. */
>> -    tmpDirInSandbox = useChroot ? canonPath("/tmp", true) + "/guix-build-" + drvName + "-0" : tmpDir;
>> +    tmpDirInSandbox = useChroot ? canonPath("/tmp", true) + "/guix-build-" + drvName : tmpDir;
> 
> The result would be that the temporary directory would always have a
> different name inside and outside the container.  Consequently,
> debugging along the lines of what the manual suggests (info "(guix)
> Debugging Build Failures") would become pretty much impossible.

Why do you think it would become impossible?

Message #37 received at 36443 <at> debbugs.gnu.org (full text, mbox):

From: Mark H Weaver <mhw <at> netris.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 36443 <at> debbugs.gnu.org, Robert Vollmert <rob <at> vllmrt.net>
Subject: Re: bug#36443: Canonicalized build directory name in container leads
 to confusion (was guix build mixes build dirs?)
Date: Tue, 02 Jul 2019 23:48:18 -0400

Hi Ludovic,

Ludovic Courtès <ludo <at> gnu.org> wrote:
> The result would be that the temporary directory would always have a
> different name inside and outside the container.  Consequently,
> debugging along the lines of what the manual suggests (info "(guix)
> Debugging Build Failures") would become pretty much impossible.

It's a good point, but I think "pretty much impossible" is an
exaggeration, because you can simply rename the failed build directory
from /tmp/guix-build-*.drv-0 to /tmp/guix-build-*.drv before debugging.

However, it's true that this would be a slightly annoying extra step
that is currently not required in the common case.

> Thus, I'm in favor of the status quo.

I don't feel strongly either way.  At first it seemed like a clear
improvement, but now I'm unsure.

     Thanks,
       Mark

Message #40 received at 36443 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Robert Vollmert <rob <at> vllmrt.net>
Cc: Mark H Weaver <mhw <at> netris.org>, 36443 <at> debbugs.gnu.org
Subject: Re: bug#36443: Canonicalized build directory name in container leads
 to confusion (was guix build mixes build dirs?)
Date: Thu, 04 Jul 2019 09:59:47 +0200

Hi,

Robert Vollmert <rob <at> vllmrt.net> skribis:

>> On 2. Jul 2019, at 15:37, Ludovic Courtès <ludo <at> gnu.org> wrote:
>> 
>>>     /* In a sandbox, for determinism, always use the same temporary
>>>        directory. */
>>> -    tmpDirInSandbox = useChroot ? canonPath("/tmp", true) + "/guix-build-" + drvName + "-0" : tmpDir;
>>> +    tmpDirInSandbox = useChroot ? canonPath("/tmp", true) + "/guix-build-" + drvName : tmpDir;
>> 
>> The result would be that the temporary directory would always have a
>> different name inside and outside the container.  Consequently,
>> debugging along the lines of what the manual suggests (info "(guix)
>> Debugging Build Failures") would become pretty much impossible.
>
> Why do you think it would become impossible?

“Impossible” is an exaggeration, but when you source the
‘environment-variables’ file, for example, PWD and other variables will
refer to /tmp/guix-build-….drv, which won’t exist.  Likewise, generated
files such as Makefiles would have captured the ….drv name.

Like Mark writes, it’s not the end of the world: you can simply rename
/tmp/guix-build-….drv-0 to /tmp/guix-build-….drv.  However, it means
that things would be inconvenient by default, which doesn’t sound great
to me.

WDYT?

Thanks,
Ludo’.

Message #43 received at 36443 <at> debbugs.gnu.org (full text, mbox):

From: Robert Vollmert <rob <at> vllmrt.net>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Mark H Weaver <mhw <at> netris.org>, 36443 <at> debbugs.gnu.org
Subject: Re: bug#36443: Canonicalized build directory name in container leads
 to confusion (was guix build mixes build dirs?)
Date: Thu, 4 Jul 2019 10:04:13 +0200

> On 4. Jul 2019, at 09:59, Ludovic Courtès <ludo <at> gnu.org> wrote:
> 
> Hi,
> 
> “Impossible” is an exaggeration, but when you source the
> ‘environment-variables’ file, for example, PWD and other variables will
> refer to /tmp/guix-build-….drv, which won’t exist.  Likewise, generated
> files such as Makefiles would have captured the ….drv name.

But, wait, won’t they refer to /tmp/guix-build-0.drv? So debugging a build
from /tmp/guix-build-1.drv will use a mix of both directories?

> Like Mark writes, it’s not the end of the world: you can simply rename
> /tmp/guix-build-….drv-0 to /tmp/guix-build-….drv.  However, it means
> that things would be inconvenient by default, which doesn’t sound great
> to me.
> 
> WDYT?

I don't particularly care anymore. I think it’s a confusing mess, but for
myself I’ve learnt this wart and won’t run into the problem anymore.

Cheers
Robert

Message #48 received at 36443-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Robert Vollmert <rob <at> vllmrt.net>
Cc: Mark H Weaver <mhw <at> netris.org>, 36443-done <at> debbugs.gnu.org
Subject: Re: bug#36443: Canonicalized build directory name in container leads
 to confusion (was guix build mixes build dirs?)
Date: Thu, 04 Jul 2019 17:12:32 +0200

Hello,

Robert Vollmert <rob <at> vllmrt.net> skribis:

>> On 4. Jul 2019, at 09:59, Ludovic Courtès <ludo <at> gnu.org> wrote:
>> 
>> Hi,
>> 
>> “Impossible” is an exaggeration, but when you source the
>> ‘environment-variables’ file, for example, PWD and other variables will
>> refer to /tmp/guix-build-….drv, which won’t exist.  Likewise, generated
>> files such as Makefiles would have captured the ….drv name.
>
> But, wait, won’t they refer to /tmp/guix-build-0.drv? So debugging a build
> from /tmp/guix-build-1.drv will use a mix of both directories?

As the manual explains, the name inside the container is fixed, it’s
always .drv-0; the name outside may vary to avoid naming collisions or
as a consequence of setting TMPDIR.

>> Like Mark writes, it’s not the end of the world: you can simply rename
>> /tmp/guix-build-….drv-0 to /tmp/guix-build-….drv.  However, it means
>> that things would be inconvenient by default, which doesn’t sound great
>> to me.
>> 
>> WDYT?
>
> I don't particularly care anymore. I think it’s a confusing mess, but for
> myself I’ve learnt this wart and won’t run into the problem anymore.

I understand this has been a confusing experience, but I can’t really
think of any other sensible way to approach it.

Anyway, closing it now.  Thank you!

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.