GNU bug report logs - #37384
Daemon does not honor “useSubstitutes” on armhf

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 37384 in the body.
You can then email your comments to 37384 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Timothy Sample <samplet <at> ngyro.com>
To: bug-guix <at> gnu.org
Subject: Daemon does not honor “useSubstitutes”
 on armhf
Date: Wed, 11 Sep 2019 14:51:07 -0400

Currently, the “guix” package cannot build on armhf due to test
failures: <https://ci.guix.gnu.org/build/1705321/details>.  (I can
confirm this on my local machine, too.)

It looks like there is some confusion about the layout of the “Settings”
struct.  At least, if I add some print statements in the code to print
the address of the “useSubstitutes” flag:

    printMsg(lvlError, format("XXX: %1%") % &settings.useSubstitutes);

it gives me different addresses depending on whether I’m printing from
the main daemon loop or from the “querySubstitutablePathInfos” method.
The addresses suspiciously differ by eight.

I can confirm that the addresses are the same on x86_64.

My guess is that this bug was revealed by the changes made in commit
f6919ebdc6b0ce0286814cc6ab0564b1a4c67f5f (making the daemon assume a
single substituter).  Before this commit, it looks like the daemon would
give up because there were no substituters.  Now it relies on checking
“useSubstitutes”, which is not in fact reliable.  I’ve tested everything
from before this commit, and the addresses are still different.

Weirdly, in GDB, I can be right before the following line:

    if (!settings.useSubstitutes) return;

run “p settings.useSubstitutes” and see “false”, and then continue
running through the method without hitting that “return”.  It seems GDB
is not confused about the flag, just the actual code.

Looking at the disassembly was no use to me, since I don’t know much
about ARM, and it was not simple enough for me to guess what it was
doing.


-- Tim

Message #8 received at 37384 <at> debbugs.gnu.org (full text, mbox):

From: Timothy Sample <samplet <at> ngyro.com>
To: 37384 <at> debbugs.gnu.org
Subject: Re: bug#37384: Daemon does not honor “useSubstitutes” on armhf
Date: Wed, 11 Sep 2019 16:28:22 -0400

Timothy Sample <samplet <at> ngyro.com> writes:

> it gives me different addresses depending on whether I’m printing from
> the main daemon loop or from the “querySubstitutablePathInfos” method.
> The addresses suspiciously differ by eight.

The problem goes away if I move “useSubstitutes” above “reservedSize”,
which has type “off_t”.  I’m guessing this is because of something fishy
with “_FILE_OFFSET_BITS”, but I’m not sure what just yet.


-- Tim

Message #11 received at 37384 <at> debbugs.gnu.org (full text, mbox):

From: Timothy Sample <samplet <at> ngyro.com>
To: 37384 <at> debbugs.gnu.org
Subject: Re: bug#37384: Daemon does not honor “useSubstitutes” on armhf
Date: Thu, 12 Sep 2019 01:47:51 -0400

[Message part 1 (text/plain, inline)]

Timothy Sample <samplet <at> ngyro.com> writes:

> The problem goes away if I move “useSubstitutes” above “reservedSize”,
> which has type “off_t”.  I’m guessing this is because of something fishy
> with “_FILE_OFFSET_BITS”, but I’m not sure what just yet.

Here’s my best guess as to what is going on.  In “nix-daemon.cc” the
order of includes causes “off_t” to be defined before
“_FILE_OFFSET_BITS”, which results in it being something like a 32-bit
signed integer.  In “local-store.cc”, “_FILE_OFFSET_BITS” gets set
first, yielding a 64-bit “off_t”.  This causes all of the addresses in
the “Settings” struct to be mismatched after “reservedSize”.  I plugged
in “sizeof(off_t)” into the “printMsg” calls from before, and (lo and
behold) I saw 4 from “nix-daemon.cc” and 8 from “local-store.cc”.

The following patch fixes the problem:

[0001-daemon-Include-config.h-in-nix-daemon.cc.patch (text/x-patch, inline)]

From 3c3eafac82e0a6e8a37363d6eb46f128e585705a Mon Sep 17 00:00:00 2001
From: Timothy Sample <samplet <at> ngyro.com>
Date: Thu, 12 Sep 2019 00:50:54 -0400
Subject: [PATCH] daemon: Include 'config.h' in 'nix-daemon.cc'.

* nix/nix-daemon/nix-daemon.cc: Include 'config.h'.
---
 nix/nix-daemon/nix-daemon.cc | 1 +
 1 file changed, 1 insertion(+)

diff --git a/nix/nix-daemon/nix-daemon.cc b/nix/nix-daemon/nix-daemon.cc
index ffac6cde34..1163a249d1 100644
--- a/nix/nix-daemon/nix-daemon.cc
+++ b/nix/nix-daemon/nix-daemon.cc
@@ -1,3 +1,4 @@
+#include "config.h"
 #include "shared.hh"
 #include "local-store.hh"
 #include "util.hh"
-- 
2.23.0

[Message part 3 (text/plain, inline)]

Is this okay to push?  It seems kind of conspicuous that it’s missing in
the first place.  Is it missing for a good reason?


-- Tim

Message #14 received at 37384 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Timothy Sample <samplet <at> ngyro.com>
Cc: 37384 <at> debbugs.gnu.org
Subject: Re: bug#37384: Daemon does not honor “useSubstitutes” on armhf
Date: Mon, 16 Sep 2019 17:52:59 +0200

Hello Timothy,

Timothy Sample <samplet <at> ngyro.com> skribis:

> The following patch fixes the problem:
>
> From 3c3eafac82e0a6e8a37363d6eb46f128e585705a Mon Sep 17 00:00:00 2001
> From: Timothy Sample <samplet <at> ngyro.com>
> Date: Thu, 12 Sep 2019 00:50:54 -0400
> Subject: [PATCH] daemon: Include 'config.h' in 'nix-daemon.cc'.
>
> * nix/nix-daemon/nix-daemon.cc: Include 'config.h'.
> ---
>  nix/nix-daemon/nix-daemon.cc | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/nix/nix-daemon/nix-daemon.cc b/nix/nix-daemon/nix-daemon.cc
> index ffac6cde34..1163a249d1 100644
> --- a/nix/nix-daemon/nix-daemon.cc
> +++ b/nix/nix-daemon/nix-daemon.cc
> @@ -1,3 +1,4 @@
> +#include "config.h"
>  #include "shared.hh"
>  #include "local-store.hh"
>  #include "util.hh"

LGTM.

Thanks for the debugging effort that led to this fix!

Ludo’.

Message #17 received at 37384 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Mikhail Kryshen <mikhail <at> kryshen.net>
Cc: 37426 <at> debbugs.gnu.org, 37384 <at> debbugs.gnu.org,
 Timothy Sample <samplet <at> ngyro.com>
Subject: Re: bug#37426: guix-1.0.1-5.cc98b00 fails tests on i686-linux
Date: Mon, 16 Sep 2019 23:06:14 +0200

Hi Mikhail,

Mikhail Kryshen <mikhail <at> kryshen.net> skribis:

> http://ci.guix.gnu.org/log/jkgrd9sv605jj3l819wp1jba99axfl0g-guix-1.0.1-5.cc98b00
>
> This breaks system configuration on i686 with current Guix.

Thanks for the heads-up.

The test failures are:

--8<---------------cut here---------------start------------->8---
test-name: substitutable-path-info when substitutes are turned off
location: /tmp/guix-build-guix-1.0.1-5.cc98b00.drv-0/source/tests/store.scm:310
source:
+ (test-equal
+   "substitutable-path-info when substitutes are turned off"
+   '()
+   (with-store
+     s
+     (set-build-options s #:use-substitutes? #f)
+     (let* ((b (add-to-store
+                 s
+                 "bash"
+                 #t
+                 "sha256"
+                 (search-bootstrap-binary
+                   "bash"
+                   (%current-system))))
+            (d (derivation
+                 s
+                 "the-thing"
+                 b
+                 '("--version")
+                 #:inputs
+                 `((,b))))
+            (o (derivation->output-path d)))
+       (with-derivation-narinfo
+         d
+         (substitutable-path-info s (list o))))))
expected-value: ()
actual-value: (#<<substitutable> path: "/tmp/guix-tests/store/k8yr3vwh9lry8r5x53p1fcpk1d78v4y3-the-thing" deriver: "/tmp/guix-tests/store/1kklp6bxdl34k0qxg4q4wxab7viyl00n-the-thing.drv" refs: () dl-size: 0 nar-size: 1234>)
result: FAIL

test-name: substitutable-paths when substitutes are turned off
location: /tmp/guix-build-guix-1.0.1-5.cc98b00.drv-0/source/tests/store.scm:323
source:
+ (test-equal
+   "substitutable-paths when substitutes are turned off"
+   '()
+   (with-store
+     s
+     (set-build-options s #:use-substitutes? #f)
+     (let* ((b (add-to-store
+                 s
+                 "bash"
+                 #t
+                 "sha256"
+                 (search-bootstrap-binary
+                   "bash"
+                   (%current-system))))
+            (d (derivation
+                 s
+                 "the-thing"
+                 b
+                 '("--version")
+                 #:inputs
+                 `((,b))))
+            warning: in 'the-thing': deprecated 'derivation' calling convention used
warning: in 'the-thing': deprecated 'derivation' calling convention used
substitute: warning: authentication and authorization of substitutes disabled!
substitute: guix substitute: warning: ACL for archive imports seems to be uninitialized, substitutes may be unavailable
warning: in 'the-thing': deprecated 'derivation' calling convention used
warning: in 'the-thing': deprecated 'derivation' calling convention used
warning: in 'the-thing': deprecated 'derivation' calling convention used
warning: in 'the-thing': deprecated 'derivation' calling convention used
substitute: warning: authentication and authorization of substitutes disabled!
substitute: guix substitute: warning: ACL for archive imports seems to be uninitialized, substitutes may be unavailable
@ build-started /tmp/guix-tests/store/fh0yssxmiv089bpylfl4v5z4qqq89p56-the-thing.drv - i686-linux /tmp/guix-tests/var/log/guix/drvs/fh//0yssxmiv089bpylfl4v5z4qqq89p56-the-thing.drv.bz2 31208
@ build-succeeded /tmp/guix-tests/store/fh0yssxmiv089bpylfl4v5z4qqq89p56-the-thing.drv -
(o (derivation->output-path d)))
+       (with-derivation-narinfo
+         d
+         (substitutable-paths s (list o))))))
expected-value: ()
actual-value: ("/tmp/guix-tests/store/k8yr3vwh9lry8r5x53p1fcpk1d78v4y3-the-thing")
result: FAIL
--8<---------------cut here---------------end--------------->8---

This may well be due to <https://issues.guix.gnu.org/issue/37384>.

When Timothy pushes the fix, I’ll update the ‘guix’ package and check
whether it builds on i686-linux.

Ludo’.

Message #22 received at 37384-done <at> debbugs.gnu.org (full text, mbox):

From: Timothy Sample <samplet <at> ngyro.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 37384-done <at> debbugs.gnu.org
Subject: Re: bug#37384: Daemon does not honor “useSubstitutes” on armhf
Date: Tue, 17 Sep 2019 00:14:42 -0400

Hi,

Ludovic Courtès <ludo <at> gnu.org> writes:

> LGTM.

Great!  I just pushed it.  Thanks for the review.


-- Tim

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.