GNU bug report logs - #19816
'guix environment' always downloads all the outputs of dependencies

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 19816 in the body.
You can then email your comments to 19816 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: bug-guix <at> gnu.org
Cc: Dave Thompson <davet <at> gnu.org>
Subject: 'guix environment' always downloads all the outputs of dependencies
Date: Sun, 08 Feb 2015 19:24:37 +0100

From <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=19641#16>:

There’s another problem, though.  When a dependency is a multiple-output
package, all its outputs are added to the environment, because
‘package->transitive-inputs’ discards the information of which output is
needed.

So for instance, both the ‘out’ and the ‘debug’ output of Coreutils end
up being downloaded and added to the environment, even though only ‘out’
is an input.

Now, the problem is that ‘build-derivations’ can only build *all* the
outputs of the given derivation.  This could be worked around either:

  1. by creating a “sink” derivation, for instance with
     ‘profile-derivation’, that could refer precisely to the output(s)
     needed; not ideal.

  2. by using (build-things (list "/the/output/path")) and resorting to
     ‘build-derivations’ only if the ‘build-things’ call did nothing
     (when passed a non-.drv store item, ‘build-things’ tries to
     substitute and does nothing if that fails.)

Ludo’.

Message #8 received at 19816 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Dave Thompson <davet <at> gnu.org>
Cc: 19816 <at> debbugs.gnu.org
Subject: Re: bug#19816: 'guix environment' always downloads all the outputs of
 dependencies
Date: Sun, 20 Dec 2015 23:34:48 +0100

ludo <at> gnu.org (Ludovic Courtès) skribis:

>>From <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=19641#16>:
>
> There’s another problem, though.  When a dependency is a multiple-output
> package, all its outputs are added to the environment, because
> ‘package->transitive-inputs’ discards the information of which output is
> needed.
>
> So for instance, both the ‘out’ and the ‘debug’ output of Coreutils end
> up being downloaded and added to the environment, even though only ‘out’
> is an input.
>
> Now, the problem is that ‘build-derivations’ can only build *all* the
> outputs of the given derivation.  This could be worked around either:
>
>   1. by creating a “sink” derivation, for instance with
>      ‘profile-derivation’, that could refer precisely to the output(s)
>      needed; not ideal.

David, I think you had started looking at doing exactly this.  Do you
have a preliminary patch you’d like to post?

I understand you may be busy preparing the Lisp Game Jam now and I
wouldn’t want to distract you from that.  ;-)  If you have something to
share, I’m happy to help start from there though (says the guy who sees
its ‘guix environment’ pointlessly downloading “debug” and “doc” outputs
again.)

Thanks,
Ludo’.

Message #11 received at 19816 <at> debbugs.gnu.org (full text, mbox):

From: "Thompson, David" <dthompson2 <at> worcester.edu>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 19816 <at> debbugs.gnu.org, Dave Thompson <davet <at> gnu.org>
Subject: Re: bug#19816: 'guix environment' always downloads all the outputs of
 dependencies
Date: Mon, 21 Dec 2015 12:21:38 -0500

On Sun, Dec 20, 2015 at 5:34 PM, Ludovic Courtès <ludo <at> gnu.org> wrote:
> ludo <at> gnu.org (Ludovic Courtès) skribis:
>
>>>From <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=19641#16>:
>>
>> There’s another problem, though.  When a dependency is a multiple-output
>> package, all its outputs are added to the environment, because
>> ‘package->transitive-inputs’ discards the information of which output is
>> needed.
>>
>> So for instance, both the ‘out’ and the ‘debug’ output of Coreutils end
>> up being downloaded and added to the environment, even though only ‘out’
>> is an input.
>>
>> Now, the problem is that ‘build-derivations’ can only build *all* the
>> outputs of the given derivation.  This could be worked around either:
>>
>>   1. by creating a “sink” derivation, for instance with
>>      ‘profile-derivation’, that could refer precisely to the output(s)
>>      needed; not ideal.
>
> David, I think you had started looking at doing exactly this.  Do you
> have a preliminary patch you’d like to post?
>
> I understand you may be busy preparing the Lisp Game Jam now and I
> wouldn’t want to distract you from that.  ;-)  If you have something to
> share, I’m happy to help start from there though (says the guy who sees
> its ‘guix environment’ pointlessly downloading “debug” and “doc” outputs
> again.)

The wip-environment-profiles branch has the code I've written thus
far.  I rebased it on master and resolved some conflicts in hopefully
the right way. :)

When I last hacked on it I got into a fully working state AFAICT, but
the tests needed updating and that proved to be the most challenging
part.  If you're willing to finish it up, that would be awesome!

Thanks,

- Dave

Message #14 received at 19816 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: "Thompson\, David" <dthompson2 <at> worcester.edu>
Cc: 19816 <at> debbugs.gnu.org, Dave Thompson <davet <at> gnu.org>
Subject: Re: bug#19816: 'guix environment' always downloads all the outputs of
 dependencies
Date: Mon, 21 Dec 2015 22:31:53 +0100

"Thompson, David" <dthompson2 <at> worcester.edu> skribis:

> On Sun, Dec 20, 2015 at 5:34 PM, Ludovic Courtès <ludo <at> gnu.org> wrote:
>> ludo <at> gnu.org (Ludovic Courtès) skribis:
>>
>>>>From <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=19641#16>:
>>>
>>> There’s another problem, though.  When a dependency is a multiple-output
>>> package, all its outputs are added to the environment, because
>>> ‘package->transitive-inputs’ discards the information of which output is
>>> needed.
>>>
>>> So for instance, both the ‘out’ and the ‘debug’ output of Coreutils end
>>> up being downloaded and added to the environment, even though only ‘out’
>>> is an input.
>>>
>>> Now, the problem is that ‘build-derivations’ can only build *all* the
>>> outputs of the given derivation.  This could be worked around either:
>>>
>>>   1. by creating a “sink” derivation, for instance with
>>>      ‘profile-derivation’, that could refer precisely to the output(s)
>>>      needed; not ideal.
>>
>> David, I think you had started looking at doing exactly this.  Do you
>> have a preliminary patch you’d like to post?
>>
>> I understand you may be busy preparing the Lisp Game Jam now and I
>> wouldn’t want to distract you from that.  ;-)  If you have something to
>> share, I’m happy to help start from there though (says the guy who sees
>> its ‘guix environment’ pointlessly downloading “debug” and “doc” outputs
>> again.)
>
> The wip-environment-profiles branch has the code I've written thus
> far.  I rebased it on master and resolved some conflicts in hopefully
> the right way. :)
>
> When I last hacked on it I got into a fully working state AFAICT, but
> the tests needed updating and that proved to be the most challenging
> part.  If you're willing to finish it up, that would be awesome!

Excellent, I had forgotten there was already a branch.

I’ll see what I can do before or after my vacations!

Ludo’.

Message #19 received at 19816-done <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: "Thompson\, David" <dthompson2 <at> worcester.edu>
Cc: 19816-done <at> debbugs.gnu.org, Dave Thompson <davet <at> gnu.org>
Subject: Re: bug#19816: 'guix environment' always downloads all the outputs of
 dependencies
Date: Fri, 12 Feb 2016 22:07:31 +0100

"Thompson, David" <dthompson2 <at> worcester.edu> skribis:

> On Sun, Dec 20, 2015 at 5:34 PM, Ludovic Courtès <ludo <at> gnu.org> wrote:
>> ludo <at> gnu.org (Ludovic Courtès) skribis:
>>
>>>>>From <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=19641#16>:
>>>
>>> There’s another problem, though.  When a dependency is a multiple-output
>>> package, all its outputs are added to the environment, because
>>> ‘package->transitive-inputs’ discards the information of which output is
>>> needed.
>>>
>>> So for instance, both the ‘out’ and the ‘debug’ output of Coreutils end
>>> up being downloaded and added to the environment, even though only ‘out’
>>> is an input.
>>>
>>> Now, the problem is that ‘build-derivations’ can only build *all* the
>>> outputs of the given derivation.  This could be worked around either:
>>>
>>>   1. by creating a “sink” derivation, for instance with
>>>      ‘profile-derivation’, that could refer precisely to the output(s)
>>>      needed; not ideal.
>>
>> David, I think you had started looking at doing exactly this.  Do you
>> have a preliminary patch you’d like to post?
>>
>> I understand you may be busy preparing the Lisp Game Jam now and I
>> wouldn’t want to distract you from that.  ;-)  If you have something to
>> share, I’m happy to help start from there though (says the guy who sees
>> its ‘guix environment’ pointlessly downloading “debug” and “doc” outputs
>> again.)
>
> The wip-environment-profiles branch has the code I've written thus
> far.  I rebased it on master and resolved some conflicts in hopefully
> the right way. :)
>
> When I last hacked on it I got into a fully working state AFAICT, but
> the tests needed updating and that proved to be the most challenging
> part.  If you're willing to finish it up, that would be awesome!

It took me a while but I’ve finally finished it and pushed as 779aa00.

I took a different strategy for the tests, which is to rely (in part) on
the output of ‘guix gc --references’ to determine whether the profile
contains all we need.

Thanks,
Ludo’.

Message #22 received at 19816-done <at> debbugs.gnu.org (full text, mbox):

From: "Thompson, David" <dthompson2 <at> worcester.edu>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 19816-done <at> debbugs.gnu.org, Dave Thompson <davet <at> gnu.org>
Subject: Re: bug#19816: 'guix environment' always downloads all the outputs of
 dependencies
Date: Fri, 12 Feb 2016 16:37:38 -0500

On Fri, Feb 12, 2016 at 4:07 PM, Ludovic Courtès <ludo <at> gnu.org> wrote:
> "Thompson, David" <dthompson2 <at> worcester.edu> skribis:
>
>> On Sun, Dec 20, 2015 at 5:34 PM, Ludovic Courtès <ludo <at> gnu.org> wrote:
>>> ludo <at> gnu.org (Ludovic Courtès) skribis:
>>>
>>>>>>From <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=19641#16>:
>>>>
>>>> There’s another problem, though.  When a dependency is a multiple-output
>>>> package, all its outputs are added to the environment, because
>>>> ‘package->transitive-inputs’ discards the information of which output is
>>>> needed.
>>>>
>>>> So for instance, both the ‘out’ and the ‘debug’ output of Coreutils end
>>>> up being downloaded and added to the environment, even though only ‘out’
>>>> is an input.
>>>>
>>>> Now, the problem is that ‘build-derivations’ can only build *all* the
>>>> outputs of the given derivation.  This could be worked around either:
>>>>
>>>>   1. by creating a “sink” derivation, for instance with
>>>>      ‘profile-derivation’, that could refer precisely to the output(s)
>>>>      needed; not ideal.
>>>
>>> David, I think you had started looking at doing exactly this.  Do you
>>> have a preliminary patch you’d like to post?
>>>
>>> I understand you may be busy preparing the Lisp Game Jam now and I
>>> wouldn’t want to distract you from that.  ;-)  If you have something to
>>> share, I’m happy to help start from there though (says the guy who sees
>>> its ‘guix environment’ pointlessly downloading “debug” and “doc” outputs
>>> again.)
>>
>> The wip-environment-profiles branch has the code I've written thus
>> far.  I rebased it on master and resolved some conflicts in hopefully
>> the right way. :)
>>
>> When I last hacked on it I got into a fully working state AFAICT, but
>> the tests needed updating and that proved to be the most challenging
>> part.  If you're willing to finish it up, that would be awesome!
>
> It took me a while but I’ve finally finished it and pushed as 779aa00.
>
> I took a different strategy for the tests, which is to rely (in part) on
> the output of ‘guix gc --references’ to determine whether the profile
> contains all we need.

Thank you!  I'm excited that this is finally in master.

- Dave

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.