GNU bug report logs - #41406
Docker package retains Go references (was Re: Guix closure size of a system?)

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 41406 in the body.
You can then email your comments to 41406 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: bug-guix <at> gnu.org
Cc: Pierre Neidhardt <mail <at> ambrevar.xyz>
Subject: Docker package retains Go references (was Re: Guix closure size of a
 system?)
Date: Tue, 19 May 2020 13:27:40 -0400

The Docker package should not refer to the Go compiler after it is
built:

On Tue, May 19, 2020 at 12:22:28PM -0400, Leo Famulari wrote:
> On Tue, May 19, 2020 at 05:58:54PM +0200, Pierre Neidhardt wrote:
> > I haven't looked at the details, but
> > 
> > --8<---------------cut here---------------start------------->8---
> > > guix size docker
> > store item                                                       total    self
> > /gnu/store/w8gjhcw6a16rk1dvxa97bz2znal5fihm-go-1.13.9              639.1   350.7  28.2%
> > --8<---------------cut here---------------end--------------->8---
> > 
> > Then
> > 
> > --8<---------------cut here---------------start------------->8---
> > > grep w8gjhcw6a16rk1dvxa97bz2znal5fihm dockerd
> > Binary file dockerd matches
> > --8<---------------cut here---------------end--------------->8---
> 
> I don't think that package is using the go-build-system, so that is
> probably the reason for the bogus reference.

Message #8 received at 41406 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: Leo Famulari <leo <at> famulari.name>
Cc: 41406 <at> debbugs.gnu.org, Pierre Neidhardt <mail <at> ambrevar.xyz>
Subject: Re: bug#41406: Docker package retains Go references (was Re: Guix
 closure size of a system?)
Date: Tue, 19 May 2020 20:16:41 +0200

[Message part 1 (text/plain, inline)]

Hi Leo,

the patch below makes the grep return nothing, but guix size still contains go.

diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm
index c95ca3f9d7..bf009a792b 100644
--- a/gnu/packages/docker.scm
+++ b/gnu/packages/docker.scm
@@ -544,7 +544,9 @@ built-in registry server of Docker.")
                (install-file (string-append "bundles/dynbinary-daemon/dockerd-"
                                             (getenv "VERSION"))
                              out-bin)
-               #t))))))
+               #t)))
+         (add-after 'install 'remove-go-references
+           (assoc-ref go:%standard-phases 'remove-go-references)))))
     (inputs
      `(("btrfs-progs" ,btrfs-progs)
        ("containerd" ,containerd)       ; for containerd-shim

[Message part 2 (application/pgp-signature, inline)]

Message #11 received at 41406 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 41406 <at> debbugs.gnu.org, Pierre Neidhardt <mail <at> ambrevar.xyz>
Subject: Re: bug#41406: Docker package retains Go references (was Re: Guix
 closure size of a system?)
Date: Tue, 19 May 2020 14:25:23 -0400

[Message part 1 (text/plain, inline)]

On Tue, May 19, 2020 at 08:16:41PM +0200, Danny Milosavljevic wrote:
> Hi Leo,
> 
> the patch below makes the grep return nothing, but guix size still contains go.

What about `guix gc --references /gnu/store/...-docker`?

> 
> diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm
> index c95ca3f9d7..bf009a792b 100644
> --- a/gnu/packages/docker.scm
> +++ b/gnu/packages/docker.scm
> @@ -544,7 +544,9 @@ built-in registry server of Docker.")
>                 (install-file (string-append "bundles/dynbinary-daemon/dockerd-"
>                                              (getenv "VERSION"))
>                               out-bin)
> -               #t))))))
> +               #t)))
> +         (add-after 'install 'remove-go-references
> +           (assoc-ref go:%standard-phases 'remove-go-references)))))
>      (inputs
>       `(("btrfs-progs" ,btrfs-progs)
>         ("containerd" ,containerd)       ; for containerd-shim
> 

[signature.asc (application/pgp-signature, inline)]

Message #14 received at 41406 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: Leo Famulari <leo <at> famulari.name>
Cc: 41406 <at> debbugs.gnu.org, Pierre Neidhardt <mail <at> ambrevar.xyz>
Subject: Re: bug#41406: Docker package retains Go references (was Re: Guix
 closure size of a system?)
Date: Tue, 19 May 2020 20:32:34 +0200

[Message part 1 (text/plain, inline)]

On Tue, 19 May 2020 14:25:23 -0400
Leo Famulari <leo <at> famulari.name> wrote:

> guix gc --references

Doesn't contain go.

[Message part 2 (application/pgp-signature, inline)]

Message #17 received at 41406 <at> debbugs.gnu.org (full text, mbox):

From: Julien Lepiller <julien <at> lepiller.eu>
To: Danny Milosavljevic <dannym <at> scratchpost.org>,
 Leo Famulari <leo <at> famulari.name>
Cc: 41406 <at> debbugs.gnu.org, Pierre Neidhardt <mail <at> ambrevar.xyz>
Subject: Re: bug#41406: Docker package retains Go references (was Re: Guix
 closure size of a system?)
Date: Tue, 19 May 2020 14:43:50 -0400

Le 19 mai 2020 14:32:34 GMT-04:00, Danny Milosavljevic <dannym <at> scratchpost.org> a écrit :
>On Tue, 19 May 2020 14:25:23 -0400
>Leo Famulari <leo <at> famulari.name> wrote:
>
>> guix gc --references
>
>Doesn't contain go.

You can try guix graph -t references --path /gnu/store/…-docker /gnu/store/…-go

That should tell you which package references go and how it gets in the closure of docker.

Message #20 received at 41406 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 41406 <at> debbugs.gnu.org, Pierre Neidhardt <mail <at> ambrevar.xyz>
Subject: Re: bug#41406: Docker package retains Go references (was Re: Guix
 closure size of a system?)
Date: Tue, 19 May 2020 15:00:39 -0400

[Message part 1 (text/plain, inline)]

On Tue, May 19, 2020 at 08:32:34PM +0200, Danny Milosavljevic wrote:
> On Tue, 19 May 2020 14:25:23 -0400
> Leo Famulari <leo <at> famulari.name> wrote:
> 
> > guix gc --references
> 
> Doesn't contain go.

Then your patch works!

[signature.asc (application/pgp-signature, inline)]

Message #25 received at 41406-done <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: Leo Famulari <leo <at> famulari.name>
Cc: 41406-done <at> debbugs.gnu.org, Pierre Neidhardt <mail <at> ambrevar.xyz>
Subject: Re: bug#41406: Docker package retains Go references (was Re: Guix
 closure size of a system?)
Date: Tue, 19 May 2020 21:05:37 +0200

[Message part 1 (text/plain, inline)]

Hi Leo,

> Then your patch works!

ok!

Pushed to guix master as commit f523eaba7b5460937a230a824550243d75e5f1dc.

[Message part 2 (application/pgp-signature, inline)]

Message #28 received at 41406 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Neidhardt <mail <at> ambrevar.xyz>
To: Danny Milosavljevic <dannym <at> scratchpost.org>,
 Leo Famulari <leo <at> famulari.name>
Cc: 41406 <at> debbugs.gnu.org
Subject: Re: bug#41406: Docker package retains Go references (was Re: Guix
 closure size of a system?)
Date: Wed, 20 May 2020 08:16:32 +0200

[Message part 1 (text/plain, inline)]

Danny Milosavljevic <dannym <at> scratchpost.org> writes:

> On Tue, 19 May 2020 14:25:23 -0400
> Leo Famulari <leo <at> famulari.name> wrote:
>
>> guix gc --references
>
> Doesn't contain go.

So why did `guix size` contain go?

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[signature.asc (application/pgp-signature, inline)]

Message #31 received at 41406 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Pierre Neidhardt <mail <at> ambrevar.xyz>
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>, 41406 <at> debbugs.gnu.org
Subject: Re: bug#41406: Docker package retains Go references (was Re: Guix
 closure size of a system?)
Date: Wed, 20 May 2020 12:43:16 -0400

On Wed, May 20, 2020 at 08:16:32AM +0200, Pierre Neidhardt wrote:
> Danny Milosavljevic <dannym <at> scratchpost.org> writes:
> 
> > On Tue, 19 May 2020 14:25:23 -0400
> > Leo Famulari <leo <at> famulari.name> wrote:
> >
> >> guix gc --references
> >
> > Doesn't contain go.
> 
> So why did `guix size` contain go?

`guix size` also accounts for build-time dependencies, and Docker is
written in Go.

Message #34 received at 41406 <at> debbugs.gnu.org (full text, mbox):

From: Marius Bakke <mbakke <at> fastmail.com>
To: Leo Famulari <leo <at> famulari.name>, Pierre Neidhardt <mail <at> ambrevar.xyz>
Cc: 41406 <at> debbugs.gnu.org
Subject: Re: bug#41406: Docker package retains Go references (was Re: Guix
 closure size of a system?)
Date: Wed, 20 May 2020 19:05:57 +0200

[Message part 1 (text/plain, inline)]

Leo Famulari <leo <at> famulari.name> writes:

> On Wed, May 20, 2020 at 08:16:32AM +0200, Pierre Neidhardt wrote:
>> Danny Milosavljevic <dannym <at> scratchpost.org> writes:
>> 
>> > On Tue, 19 May 2020 14:25:23 -0400
>> > Leo Famulari <leo <at> famulari.name> wrote:
>> >
>> >> guix gc --references
>> >
>> > Doesn't contain go.
>> 
>> So why did `guix size` contain go?
>
> `guix size` also accounts for build-time dependencies, and Docker is
> written in Go.

`guix size` only looks at the closure size.  One of Dockers dependencies
must be carrying the spurious Go reference (guix graph might help here).

[signature.asc (application/pgp-signature, inline)]

Message #37 received at 41406 <at> debbugs.gnu.org (full text, mbox):

From: Julien Lepiller <julien <at> lepiller.eu>
To: Leo Famulari <leo <at> famulari.name>,Pierre Neidhardt <mail <at> ambrevar.xyz>
Cc: 41406 <at> debbugs.gnu.org
Subject: Re: bug#41406: Docker package retains Go references (was Re: Guix
 closure size of a system?)
Date: Wed, 20 May 2020 14:30:09 -0400

Le 20 mai 2020 12:43:16 GMT-04:00, Leo Famulari <leo <at> famulari.name> a écrit :
>On Wed, May 20, 2020 at 08:16:32AM +0200, Pierre Neidhardt wrote:
>> Danny Milosavljevic <dannym <at> scratchpost.org> writes:
>> 
>> > On Tue, 19 May 2020 14:25:23 -0400
>> > Leo Famulari <leo <at> famulari.name> wrote:
>> >
>> >> guix gc --references
>> >
>> > Doesn't contain go.
>> 
>> So why did `guix size` contain go?
>
>`guix size` also accounts for build-time dependencies, and Docker is
>written in Go.

As I said:

guix graph -t references --path /gnu/store/…-docker /gnu/store/…-go

Should give you the shortest path from docker to go, looking only at references.

Message #40 received at 41406 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Neidhardt <mail <at> ambrevar.xyz>
To: Leo Famulari <leo <at> famulari.name>
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>, 41406 <at> debbugs.gnu.org
Subject: Re: bug#41406: Docker package retains Go references (was Re: Guix
 closure size of a system?)
Date: Wed, 20 May 2020 23:09:37 +0200

[Message part 1 (text/plain, inline)]

Leo Famulari <leo <at> famulari.name> writes:

> `guix size` also accounts for build-time dependencies, and Docker is
> written in Go.

Are you sure about this?  Marius seems to disagree unless I'm
misunderstanding the two statements.

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[signature.asc (application/pgp-signature, inline)]

Message #43 received at 41406 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Pierre Neidhardt <mail <at> ambrevar.xyz>
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>, 41406 <at> debbugs.gnu.org
Subject: Re: bug#41406: Docker package retains Go references (was Re: Guix
 closure size of a system?)
Date: Wed, 20 May 2020 19:01:24 -0400

[Message part 1 (text/plain, inline)]

On Wed, May 20, 2020 at 11:09:37PM +0200, Pierre Neidhardt wrote:
> Leo Famulari <leo <at> famulari.name> writes:
> 
> > `guix size` also accounts for build-time dependencies, and Docker is
> > written in Go.
> 
> Are you sure about this?  Marius seems to disagree unless I'm
> misunderstanding the two statements.

I thought I was correct but it's certainly possible I'm wrong! The
manual isn't clear IMO and I don't have time to read the code.

The `guix graph` suggestions should help get to the bottom of it,
though.

[signature.asc (application/pgp-signature, inline)]

Message #46 received at 41406 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Neidhardt <mail <at> ambrevar.xyz>
To: Leo Famulari <leo <at> famulari.name>
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>, 41406 <at> debbugs.gnu.org
Subject: Re: bug#41406: Docker package retains Go references (was Re: Guix
 closure size of a system?)
Date: Thu, 21 May 2020 09:02:36 +0200

[Message part 1 (text/plain, inline)]

Leo Famulari <leo <at> famulari.name> writes:

> I thought I was correct but it's certainly possible I'm wrong! The
> manual isn't clear IMO and I don't have time to read the code.

I remember having a similar confusion in the past.  We should make it
more explicit what `guix size` does compared to `guix gc  --references'
for instance.

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[signature.asc (application/pgp-signature, inline)]

Message #49 received at 41406 <at> debbugs.gnu.org (full text, mbox):

From: zimoun <zimon.toutoune <at> gmail.com>
To: Julien Lepiller <julien <at> lepiller.eu>
Cc: 41406 <at> debbugs.gnu.org, Pierre Neidhardt <mail <at> ambrevar.xyz>,
 Leo Famulari <leo <at> famulari.name>
Subject: Re: bug#41406: Docker package retains Go references (was Re: Guix
 closure size of a system?)
Date: Thu, 21 May 2020 11:41:54 +0200

On Wed, 20 May 2020 at 20:31, Julien Lepiller <julien <at> lepiller.eu> wrote:

> guix graph -t references --path /gnu/store/…-docker /gnu/store/…-go
>
> Should give you the shortest path from docker to go, looking only at references.

It seems that the culprit is the package 'runc'.

--8<---------------cut here---------------start------------->8---
guix graph -t references \
     /gnu/store/scmx1fphz8kphx37rbckiqvkkh125y6d-docker-19.03.7 \
     --path \
     /gnu/store/w8gjhcw6a16rk1dvxa97bz2znal5fihm-go-1.13.9

/gnu/store/scmx1fphz8kphx37rbckiqvkkh125y6d-docker-19.03.7
/gnu/store/vycnhl10jc3jv4yc7dy1p6maykipidkp-runc-1.0.0-rc6
/gnu/store/w8gjhcw6a16rk1dvxa97bz2znal5fihm-go-1.13.9
--8<---------------cut here---------------end--------------->8---

Message #52 received at 41406 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: zimoun <zimon.toutoune <at> gmail.com>
Cc: Julien Lepiller <julien <at> lepiller.eu>, Pierre Neidhardt <mail <at> ambrevar.xyz>,
 41406 <at> debbugs.gnu.org
Subject: Re: bug#41406: Docker package retains Go references (was Re: Guix
 closure size of a system?)
Date: Thu, 21 May 2020 12:20:09 -0400

On Thu, May 21, 2020 at 11:41:54AM +0200, zimoun wrote:
> On Wed, 20 May 2020 at 20:31, Julien Lepiller <julien <at> lepiller.eu> wrote:
> 
> > guix graph -t references --path /gnu/store/…-docker /gnu/store/…-go
> >
> > Should give you the shortest path from docker to go, looking only at references.
> 
> It seems that the culprit is the package 'runc'.
> 
> --8<---------------cut here---------------start------------->8---
> guix graph -t references \
>      /gnu/store/scmx1fphz8kphx37rbckiqvkkh125y6d-docker-19.03.7 \
>      --path \
>      /gnu/store/w8gjhcw6a16rk1dvxa97bz2znal5fihm-go-1.13.9
> 
> /gnu/store/scmx1fphz8kphx37rbckiqvkkh125y6d-docker-19.03.7
> /gnu/store/vycnhl10jc3jv4yc7dy1p6maykipidkp-runc-1.0.0-rc6
> /gnu/store/w8gjhcw6a16rk1dvxa97bz2znal5fihm-go-1.13.9
> --8<---------------cut here---------------end--------------->8---

We should make the linter complain about this reference.

Message #55 received at 41406 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: Leo Famulari <leo <at> famulari.name>
Cc: 41406 <at> debbugs.gnu.org, Pierre Neidhardt <mail <at> ambrevar.xyz>,
 zimoun <zimon.toutoune <at> gmail.com>
Subject: Re: bug#41406: Docker package retains Go references (was Re: Guix
 closure size of a system?)
Date: Tue, 26 May 2020 16:17:01 +0200

[Message part 1 (text/plain, inline)]

Hi Leo,

On Thu, 21 May 2020 12:20:09 -0400
Leo Famulari <leo <at> famulari.name> wrote:

> We should make the linter complain about this [go] reference.

What do you mean exactly?  How would it know, given that that was a runtime
dependency?  Would guix lint build the derivation and check for references
to go etc?  Sounds doable.

[Message part 2 (application/pgp-signature, inline)]

Message #58 received at 41406 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 41406 <at> debbugs.gnu.org, Pierre Neidhardt <mail <at> ambrevar.xyz>,
 zimoun <zimon.toutoune <at> gmail.com>
Subject: Re: bug#41406: Docker package retains Go references (was Re: Guix
 closure size of a system?)
Date: Tue, 26 May 2020 12:24:22 -0400

On Tue, May 26, 2020 at 04:17:01PM +0200, Danny Milosavljevic wrote:
> Hi Leo,
> 
> On Thu, 21 May 2020 12:20:09 -0400
> Leo Famulari <leo <at> famulari.name> wrote:
> 
> > We should make the linter complain about this [go] reference.
> 
> What do you mean exactly?  How would it know, given that that was a runtime
> dependency?  Would guix lint build the derivation and check for references
> to go etc?  Sounds doable.

I had confused this sitation with the linter's warnings about "inputs
that should be native".

It's probably not worth adding a linter check for this, because everyone
should be using the go-build-system for their Go packages, and if it
doesn't work, that's a bug in the build system, not in the package
definition :)

I don't think the linter should build things and check for spurious
references. At least, not just for this case.

Message #61 received at 41406 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: Leo Famulari <leo <at> famulari.name>
Cc: 41406 <at> debbugs.gnu.org, Pierre Neidhardt <mail <at> ambrevar.xyz>,
 zimoun <zimon.toutoune <at> gmail.com>
Subject: Re: bug#41406: Docker package retains Go references (was Re: Guix
 closure size of a system?)
Date: Wed, 27 May 2020 13:53:29 +0200

[Message part 1 (text/plain, inline)]

Hi Leo,

On Tue, 26 May 2020 12:24:22 -0400
Leo Famulari <leo <at> famulari.name> wrote:

> It's probably not worth adding a linter check for this, because everyone
> should be using the go-build-system for their Go packages, and if it
> doesn't work, that's a bug in the build system, not in the package

Hmm, but runc--the culprit here--does use go-build-system and still retains
a go reference erroneously.

[Message part 2 (application/pgp-signature, inline)]

Message #64 received at 41406 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 41406 <at> debbugs.gnu.org, Pierre Neidhardt <mail <at> ambrevar.xyz>,
 zimoun <zimon.toutoune <at> gmail.com>
Subject: Re: bug#41406: Docker package retains Go references (was Re: Guix
 closure size of a system?)
Date: Wed, 27 May 2020 13:22:32 -0400

[Message part 1 (text/plain, inline)]

On Wed, May 27, 2020 at 01:53:29PM +0200, Danny Milosavljevic wrote:
> Hi Leo,
> 
> On Tue, 26 May 2020 12:24:22 -0400
> Leo Famulari <leo <at> famulari.name> wrote:
> 
> > It's probably not worth adding a linter check for this, because everyone
> > should be using the go-build-system for their Go packages, and if it
> > doesn't work, that's a bug in the build system, not in the package
> 
> Hmm, but runc--the culprit here--does use go-build-system and still retains
> a go reference erroneously.

Yup, it's a bug!

[signature.asc (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.