GNU bug report logs - #30710
guix graph gives duplicate nodes

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 30710 in the body.
You can then email your comments to 30710 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Hartmut Goebel <h.goebel <at> crazy-compilers.com>
To: bug-guix <bug-guix <at> gnu.org>
Subject: guix graph gives duplicate nodes
Date: Mon, 5 Mar 2018 10:37:39 +0100

Hi,

"guix graph" delivers the same package with different IDs. Here is an
example with a node delivered twice. (For plasma-workspace, which I'm
working on, this package was even listed four times).

$ ./pre-inst-env guix graph -t package -b graphviz qtbase | grep
autoconf-wrapper
  "59511552" [label = "autoconf-wrapper-2.69", shape = box, fontname =
Helvetica];
  "59511744" [label = "autoconf-wrapper-2.69", shape = box, fontname =
Helvetica];

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel <at> crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Message #8 received at 30710 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Hartmut Goebel <h.goebel <at> crazy-compilers.com>
Cc: 30710 <at> debbugs.gnu.org
Subject: Re: bug#30710: guix graph gives duplicate nodes
Date: Mon, 05 Mar 2018 18:19:12 +0100

Hello,

Hartmut Goebel <h.goebel <at> crazy-compilers.com> skribis:

> "guix graph" delivers the same package with different IDs. Here is an
> example with a node delivered twice. (For plasma-workspace, which I'm
> working on, this package was even listed four times).
>
> $ ./pre-inst-env guix graph -t package -b graphviz qtbase | grep
> autoconf-wrapper
>   "59511552" [label = "autoconf-wrapper-2.69", shape = box, fontname =
> Helvetica];
>   "59511744" [label = "autoconf-wrapper-2.69", shape = box, fontname =
> Helvetica];

This is expected.  Strictly speaking, we’re talking about two different
package objects, hence the different IDs.

Now, there are cases were we have multiple package objects mapping to a
single derivation.  That’s OK.  You can use “guix graph -t bag” or
similar if you want nodes that correspond to derivations.

HTH!

Ludo’.

Message #11 received at 30710 <at> debbugs.gnu.org (full text, mbox):

From: Hartmut Goebel <h.goebel <at> crazy-compilers.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 30710 <at> debbugs.gnu.org
Subject: Re: bug#30710: guix graph gives duplicate nodes
Date: Tue, 6 Mar 2018 21:28:31 +0100

Hi,
> This is expected.  Strictly speaking, we’re talking about two different
> package objects, hence the different IDs.

I wonder

a) whether it is useful to have different graph nodes for the same package.

This is about usability of the resulting graph, esp. since this is the
default graph output format. Does it help *users* for their analysis? Or
is this some *expert* insight?

b) how there can be different package objects for the same package

To my understanding, e.g. (gnu packages base) is loaded once, defining
package object abcd once and assigning it to a variable. All packages
referring to abcd use the some package object. So there should be only
*one* package object.

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel <at> crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Message #14 received at 30710 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Hartmut Goebel <h.goebel <at> crazy-compilers.com>
Cc: 30710 <at> debbugs.gnu.org
Subject: Re: bug#30710: guix graph gives duplicate nodes
Date: Wed, 07 Mar 2018 16:18:51 +0100

Hello,

Hartmut Goebel <h.goebel <at> crazy-compilers.com> skribis:

>> This is expected.  Strictly speaking, we’re talking about two different
>> package objects, hence the different IDs.
>
> I wonder
>
> a) whether it is useful to have different graph nodes for the same package.
>
> This is about usability of the resulting graph, esp. since this is the
> default graph output format. Does it help *users* for their analysis? Or
> is this some *expert* insight?

As explained in “Invoking guix graph”, the tool provides different graph
types, each at its own abstraction level.

The package graph is high-level, but as a side-effect it has this
artifact.

To developers it’s actually useful to see the graph of package objects.
There are cases where, with functions that return packages, you would
notice that you’re generating lots of package objects for the same
underlying derivation, which is super inefficient (in particular it
defeats memoization).

Most of the time, there’s exactly one package object for each
derivation; if not, that’s usually a bug.

> b) how there can be different package objects for the same package
>
> To my understanding, e.g. (gnu packages base) is loaded once, defining
> package object abcd once and assigning it to a variable. All packages
> referring to abcd use the some package object. So there should be only
> *one* package object.

(eq? foo (package (inherit foo))) => #false

Yet, these two packages map to the very same derivation.

HTH,
Ludo’.

Message #21 received at 30710 <at> debbugs.gnu.org (full text, mbox):

From: Björn Höfling <bjoern.hoefling <at> bjoernhoefling.de>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 30710 <at> debbugs.gnu.org, Hartmut Goebel <h.goebel <at> crazy-compilers.com>
Subject: Re: bug#30710: guix graph gives duplicate nodes
Date: Fri, 9 Mar 2018 11:09:17 +0100

[Message part 1 (text/plain, inline)]

On Wed, 07 Mar 2018 16:18:51 +0100
ludo <at> gnu.org (Ludovic Courtès) wrote:

> Hello,
> 
> Hartmut Goebel <h.goebel <at> crazy-compilers.com> skribis:
> 
> >> This is expected.  Strictly speaking, we’re talking about two
> >> different package objects, hence the different IDs.  
> >
> > I wonder
> >
> > a) whether it is useful to have different graph nodes for the same
> > package.
> >
> > This is about usability of the resulting graph, esp. since this is
> > the default graph output format. Does it help *users* for their
> > analysis? Or is this some *expert* insight?  
> 
> As explained in “Invoking guix graph”, the tool provides different
> graph types, each at its own abstraction level.
> 
> The package graph is high-level, but as a side-effect it has this
> artifact.
> 
> To developers it’s actually useful to see the graph of package
> objects. There are cases where, with functions that return packages,
> you would notice that you’re generating lots of package objects for
> the same underlying derivation, which is super inefficient (in
> particular it defeats memoization).
> 
> Most of the time, there’s exactly one package object for each
> derivation; if not, that’s usually a bug.
> 
> > b) how there can be different package objects for the same package
> >
> > To my understanding, e.g. (gnu packages base) is loaded once,
> > defining package object abcd once and assigning it to a variable.
> > All packages referring to abcd use the some package object. So
> > there should be only *one* package object.  
> 
> (eq? foo (package (inherit foo))) => #false
> 
> Yet, these two packages map to the very same derivation.


This thing really took me a while to think about the graph system in
general and this concrete case.

Speaking about this concrete case: If you inspect the output of 

`guix graph qt` you find these interesting lines:

  "64168128" [label = "autoconf-wrapper-2.69", shape = box, fontname = Helvetica
  "64167936" [label = "autoconf-wrapper-2.69", shape = box, fontname = Helvetica

  "52941184" -> "64168128" [color = darkseagreen];
  "52940800" -> "64167936" [color = blue];

  "52941184" [label = "automake-1.15.1", shape = box, fontname = Helvetica];
  "52940800" [label = "libtool-2.4.6", shape = box, fontname = Helvetica];


If you look into gnu/packages/autotools.scm, you see that
autoconf-wrapper is not a package, but a package-factory:

(define* (autoconf-wrapper #:optional (autoconf autoconf))

Now the package definitions of "automake" and "libtool" each use the
same fragment of code in their native-inputs, but a different "package"
in the eq?-sense, although they basically want the same thing:

 `(("autoconf" ,(autoconf-wrapper))

As ludo stated above: "Most of the time, there’s exactly one package
object for each derivation; if not, that’s usually a bug."

This looks to me like a bug.

Correction:

(define autoconf-wrapper-default (autoconf-wrapper))

And then use this singular package as native-inputs to libtool and automake.

Furthermore, when I search:

find . -name "*.scm" -exec grep -H "autoconf-wrapper" "{}" ";" | less

I find about 10 packages that use the fabrik, but all in the default way.

So instead of:

  #:export (autoconf-wrapper))


We could just 

(define-public autoconf-wrapper-default (autoconf-wrapper))

and use that.

Or, if noone is using this fabrik, just drop that and make a normal package out of it.

WDYT? Reopen this one?

Björn

[Message part 2 (application/pgp-signature, inline)]

Message #24 received at 30710-done <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Björn Höfling <bjoern.hoefling <at> bjoernhoefling.de>
Cc: 30710-done <at> debbugs.gnu.org, Hartmut Goebel <h.goebel <at> crazy-compilers.com>
Subject: Re: bug#30710: guix graph gives duplicate nodes
Date: Fri, 09 Mar 2018 23:59:26 +0100

Björn Höfling <bjoern.hoefling <at> bjoernhoefling.de> skribis:

> If you look into gnu/packages/autotools.scm, you see that
> autoconf-wrapper is not a package, but a package-factory:
>
> (define* (autoconf-wrapper #:optional (autoconf autoconf))
>
> Now the package definitions of "automake" and "libtool" each use the
> same fragment of code in their native-inputs, but a different "package"
> in the eq?-sense, although they basically want the same thing:
>
>  `(("autoconf" ,(autoconf-wrapper))
>
> As ludo stated above: "Most of the time, there’s exactly one package
> object for each derivation; if not, that’s usually a bug."
>
> This looks to me like a bug.
>
> Correction:
>
> (define autoconf-wrapper-default (autoconf-wrapper))
>
> And then use this singular package as native-inputs to libtool and automake.
>
> Furthermore, when I search:
>
> find . -name "*.scm" -exec grep -H "autoconf-wrapper" "{}" ";" | less
>
> I find about 10 packages that use the fabrik, but all in the default way.
>
> So instead of:
>
>   #:export (autoconf-wrapper))
>
>
> We could just 
>
> (define-public autoconf-wrapper-default (autoconf-wrapper))
>
> and use that.
>
> Or, if noone is using this fabrik, just drop that and make a normal package out of it.
>
> WDYT? Reopen this one?

Good catch!  I implemented what you suggest above in commit
464f5447396fcec9b43f7eab71d5d42b522a157f.

Thank you!

Ludo’.

Message #27 received at 30710 <at> debbugs.gnu.org (full text, mbox):

From: Hartmut Goebel <h.goebel <at> crazy-compilers.com>
To: Björn Höfling <bjoern.hoefling <at> bjoernhoefling.de>,
 Ludovic Courtès <ludo <at> gnu.org>,
 30710 <30710 <at> debbugs.gnu.org>
Subject: Re: bug#30710: guix graph gives duplicate nodes
Date: Sat, 10 Mar 2018 17:31:30 +0100

On Fri, 09 Mar 2018 23:59:26 +0100 ludo <at> gnu.org (Ludovic Courtès) wrote:
>
>> Good catch!  I implemented what you suggest above in commit
>> 464f5447396fcec9b43f7eab71d5d42b522a157f.

Thanks, this solved the issue only partially: On my "kde-plasma" branch:

$ ./pre-inst-env guix graph plasma-workspace | grep autoconf
  "133094208" [label = "autoconf-wrapper-2.69", shape = box, fontname =
Helvetica];
  "133094976" [label = "autoconf-2.69", shape = box, fontname = Helvetica];
  "152772352" [label = "autoconf-wrapper-2.69", shape = box, fontname =
Helvetica];
  "152420544" [label = "autoconf-2.69", shape = box, fontname = Helvetica];

There are other packages, which are duplicate but don't have a factory
AFAIK:

$ ./pre-inst-env guix graph plasma-workspace | grep kbus
  "130257472" [label = "kdbusaddons-5.42.0", shape = box, fontname =
Helvetica];
  "148171200" [label = "kdbusaddons-5.42.0", shape = box, fontname =
Helvetica];

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel <at> crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Message #30 received at 30710 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Hartmut Goebel <h.goebel <at> crazy-compilers.com>
Cc: Björn Höfling <bjoern.hoefling <at> bjoernhoefling.de>,
 30710 <30710 <at> debbugs.gnu.org>
Subject: Re: bug#30710: guix graph gives duplicate nodes
Date: Mon, 12 Mar 2018 15:27:19 +0100

Hello,

Hartmut Goebel <h.goebel <at> crazy-compilers.com> skribis:

> On Fri, 09 Mar 2018 23:59:26 +0100 ludo <at> gnu.org (Ludovic Courtès) wrote:
>>
>>> Good catch!  I implemented what you suggest above in commit
>>> 464f5447396fcec9b43f7eab71d5d42b522a157f.
>
> Thanks, this solved the issue only partially: On my "kde-plasma" branch:
>
> $ ./pre-inst-env guix graph plasma-workspace | grep autoconf
>   "133094208" [label = "autoconf-wrapper-2.69", shape = box, fontname =
> Helvetica];
>   "133094976" [label = "autoconf-2.69", shape = box, fontname = Helvetica];
>   "152772352" [label = "autoconf-wrapper-2.69", shape = box, fontname =
> Helvetica];
>   "152420544" [label = "autoconf-2.69", shape = box, fontname = Helvetica];
>
> There are other packages, which are duplicate but don't have a factory
> AFAIK:
>
> $ ./pre-inst-env guix graph plasma-workspace | grep kbus
>   "130257472" [label = "kdbusaddons-5.42.0", shape = box, fontname =
> Helvetica];
>   "148171200" [label = "kdbusaddons-5.42.0", shape = box, fontname =
> Helvetica];

That stems from the use of ‘package-mapping’ or
‘package-input-rewriting’ (maybe via ‘package-with-python2’), which can
create distinct package objects mapping to the same derivation.

I don’t have a good solution here.

Thanks,
Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.