GNU bug report logs - #22774
Bootstrapping issue with HTTPS downloads

Previous Next

Package: guix;

Reported by: ludo <at> gnu.org (Ludovic Courtès)

Date: Mon, 22 Feb 2016 21:05:02 UTC

Severity: important

Done: ludo <at> gnu.org (Ludovic Courtès)

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 22774 in the body.
You can then email your comments to 22774 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-guix <at> gnu.org:
bug#22774; Package guix. (Mon, 22 Feb 2016 21:05:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to ludo <at> gnu.org (Ludovic Courtès):
New bug report received and forwarded. Copy sent to bug-guix <at> gnu.org. (Mon, 22 Feb 2016 21:05:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: bug-guix <at> gnu.org
Subject: Bootstrapping issue with HTTPS downloads
Date: Mon, 22 Feb 2016 22:03:54 +0100
Fixed-output derivations for HTTPS downloads depend on GnuTLS.  However,
pkg-config, one of GnuTLS’s dependencies, is now available exclusively
over HTTPS, which creates a bootstrapping issue, as reported by
Christopher W Carpenter and others:

  https://lists.gnu.org/archive/html/guix-devel/2016-02/msg00354.html

Ludo’.




Severity set to 'important' from 'normal' Request was from ludo <at> gnu.org (Ludovic Courtès) to control <at> debbugs.gnu.org. (Mon, 22 Feb 2016 21:18:02 GMT) Full text and rfc822 format available.

Information forwarded to bug-guix <at> gnu.org:
bug#22774; Package guix. (Sun, 31 Jul 2016 16:29:01 GMT) Full text and rfc822 format available.

Message #10 received at 22774 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: 22774 <at> debbugs.gnu.org
Subject: Re: bug#22774: Bootstrapping issue with HTTPS downloads
Date: Sun, 31 Jul 2016 18:28:43 +0200
ludo <at> gnu.org (Ludovic Courtès) skribis:

> Fixed-output derivations for HTTPS downloads depend on GnuTLS.  However,
> pkg-config, one of GnuTLS’s dependencies, is now available exclusively
> over HTTPS, which creates a bootstrapping issue, as reported by
> Christopher W Carpenter and others:
>
>   https://lists.gnu.org/archive/html/guix-devel/2016-02/msg00354.html

This is mitigated by the use of content-addressed mirrors over HTTP, as
in commit 40f788b9f6184436d9cc36a4dd8e7d101cd2f0ba (obviously a stop-gap
measure more than a real fix.)

Ludo’.




Information forwarded to bug-guix <at> gnu.org:
bug#22774; Package guix. (Fri, 28 Oct 2016 23:44:01 GMT) Full text and rfc822 format available.

Message #13 received at 22774 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: 22774 <at> debbugs.gnu.org
Subject: Re: bug#22774: Bootstrapping issue with HTTPS downloads
Date: Sat, 29 Oct 2016 01:43:13 +0200
ludo <at> gnu.org (Ludovic Courtès) skribis:

> Fixed-output derivations for HTTPS downloads depend on GnuTLS.  However,
> pkg-config, one of GnuTLS’s dependencies, is now available exclusively
> over HTTPS, which creates a bootstrapping issue, as reported by
> Christopher W Carpenter and others:
>
>   https://lists.gnu.org/archive/html/guix-devel/2016-02/msg00354.html

The ‘wip-oob-download’ implements “out-of-band downloads”, whereby
downloads are performed by the daemon on behalf of clients.

This works by adding a “built-in builder” in the daemon (idea stolen
from the Nix daemon): if you specify “builtin:download” as the builder
of your derivation, then guix-daemon invokes ‘guix download’ to download
the thing.  The derivation has to be fixed-output, of course.

The branch is not ready yet, notably because we’d need a way to pass
mirror lists to ‘guix download’, as an attribute of the derivation.

Another question is compatibility: what if a recent client creates a
“builtin:download” derivation on an old daemon?  The problem is that
currently old daemons do not check the builder of the .drv file until
you call ‘build-derivations’.  So I think we’ll need to bump the
protocol version so client code can check whether the daemon is new
enough.

Thoughts?

The good thing is that this should allow us to get rid of the bootstrap
binaries that are currently distributed in the source tarball of Guix
and downloaded upon ‘make’:

  https://www.gnu.org/software/guix/manual/html_node/Bootstrapping.html#Preparing-to-Use-the-Bootstrap-Binaries

This was motivated precisely by the need for an out-of-band download
mechanism for bootstrapping, which we’ll then have within guix-daemon.

Ludo’.




Information forwarded to bug-guix <at> gnu.org:
bug#22774; Package guix. (Sun, 13 Nov 2016 23:03:02 GMT) Full text and rfc822 format available.

Message #16 received at 22774 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: 22774 <at> debbugs.gnu.org
Subject: Re: bug#22774: Bootstrapping issue with HTTPS downloads
Date: Mon, 14 Nov 2016 00:02:38 +0100
ludo <at> gnu.org (Ludovic Courtès) skribis:

> The ‘wip-oob-download’ implements “out-of-band downloads”, whereby
> downloads are performed by the daemon on behalf of clients.
>
> This works by adding a “built-in builder” in the daemon (idea stolen
> from the Nix daemon): if you specify “builtin:download” as the builder
> of your derivation, then guix-daemon invokes ‘guix download’ to download
> the thing.  The derivation has to be fixed-output, of course.
>
> The branch is not ready yet, notably because we’d need a way to pass
> mirror lists to ‘guix download’, as an attribute of the derivation.

Fixed!

> Another question is compatibility: what if a recent client creates a
> “builtin:download” derivation on an old daemon?  The problem is that
> currently old daemons do not check the builder of the .drv file until
> you call ‘build-derivations’.  So I think we’ll need to bump the
> protocol version so client code can check whether the daemon is new
> enough.

To address that, I bumped the protocol number in worker-protocol.hh and
added a ‘built-in-builders’ RPC that returns the list of supported
built-in builders (a list of strings.)

That way, clients can check what the daemon supports.  For instance,
‘url-fetch’ can return either a “normal” fixed-output derivation or one
that uses “builtin:download”, depending on what the daemon supports.

Feedback welcome.

Ludo’.




Reply sent to ludo <at> gnu.org (Ludovic Courtès):
You have taken responsibility. (Wed, 16 Nov 2016 17:25:02 GMT) Full text and rfc822 format available.

Notification sent to ludo <at> gnu.org (Ludovic Courtès):
bug acknowledged by developer. (Wed, 16 Nov 2016 17:25:02 GMT) Full text and rfc822 format available.

Message #21 received at 22774-done <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: 22774-done <at> debbugs.gnu.org
Subject: Re: bug#22774: Bootstrapping issue with HTTPS downloads
Date: Wed, 16 Nov 2016 18:23:48 +0100
Hi!

I pushed the whole patch series with the fix as
05ceb8dcaf480a47cddf94ac979070b76df6556c.

The final patch series uses a new internal helper, ‘guix
perform-download’, instead of trying to bend ‘guix download’ to these
specific needs.

Ludo’.




bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Thu, 15 Dec 2016 12:24:04 GMT) Full text and rfc822 format available.

This bug report was last modified 7 years and 105 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.