GNU bug report logs - #33620
Golang programs keeping references [gnu: go: Update default to 1.11.]

Previous Next

Package: guix;

Reported by: Leo Famulari <leo <at> famulari.name>

Date: Wed, 5 Dec 2018 02:22:01 UTC

Severity: normal

Merged with 33741

Done: Leo Famulari <leo <at> famulari.name>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 33620 in the body.
You can then email your comments to 33620 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-guix <at> gnu.org:
bug#33620; Package guix. (Wed, 05 Dec 2018 02:22:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Leo Famulari <leo <at> famulari.name>:
New bug report received and forwarded. Copy sent to bug-guix <at> gnu.org. (Wed, 05 Dec 2018 02:22:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: bug-guix <at> gnu.org
Subject: Golang programs keeping references [gnu: go: Update default to 1.11.]
Date: Tue, 4 Dec 2018 21:16:01 -0500
[Message part 1 (text/plain, inline)]
----- Forwarded message from Leo Famulari <leo <at> famulari.name> -----

Date: Mon, 12 Nov 2018 12:29:25 -0500
From: Leo Famulari <leo <at> famulari.name>
To: guix-devel <at> gnu.org
Cc: Pierre Neidhardt <ambrevar <at> gmail.com>
Subject: Golang programs keeping references [gnu: go: Update default to 1.11.]
User-Agent: Mutt/1.10.1 (2018-07-13)

On Mon, Nov 12, 2018 at 04:32:46AM -0500, Pierre Neidhardt wrote:
> commit 9a65a052016572b61e3c4247fcdf9e0478656f71
> Author: Pierre Neidhardt <mail <at> ambrevar.xyz>
> Date:   Sun Nov 11 22:02:18 2018 +0100
> 
>     gnu: go: Update default to 1.11.
>     
>     * gnu/packages/golang.scm (go): Update default to 1.11.

I noticed that since this change, Go programs (that is, command-line
executables) keep references to their inputs:

$ guix gc --references $(./pre-inst-env guix build --no-grafts kurly)
/gnu/store/2b2md66fbzyspsmd5dj6zkj9hilac40r-tzdata-2018e
/gnu/store/4iwksvq53rlzphfp3xvp63ihlw226c0n-go-github-com-aki237-nscjar-0.0.0-0.e2df936
/gnu/store/5rxdjbk8h0bh1hbaan8y8ib13va2bcmw-net-base-5.3
/gnu/store/ahvdlp6y44qj6kx63rmx1sq8r61x3zc2-go-github-com-alsm-ioprogress-0.0.0-0.063c372
/gnu/store/f8yps0l8p371jgzh6cki0z5n2kgfjiwy-go-github-com-urfave-cli-1.19.1-0.934abfb
/gnu/store/l4lr0f5cjd0nbsaaf8b5dmcw1a1yypr3-glibc-2.27
/gnu/store/pp0bakrbyv9xmp1kyv2114l19s11b74z-gcc-6.4.0-lib

Previously, they did not:

$ guix gc --references $(guix build --no-grafts kurly)
/gnu/store/2b2md66fbzyspsmd5dj6zkj9hilac40r-tzdata-2018e
/gnu/store/5rxdjbk8h0bh1hbaan8y8ib13va2bcmw-net-base-5.3
/gnu/store/l4lr0f5cjd0nbsaaf8b5dmcw1a1yypr3-glibc-2.27
/gnu/store/pp0bakrbyv9xmp1kyv2114l19s11b74z-gcc-6.4.0-lib

Is this expected? I thought that Go programs were always statically
linked.



----- End forwarded message -----
[signature.asc (application/pgp-signature, inline)]

Information forwarded to bug-guix <at> gnu.org:
bug#33620; Package guix. (Sat, 15 Dec 2018 19:56:01 GMT) Full text and rfc822 format available.

Message #8 received at 33620 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: 33620 <at> debbugs.gnu.org
Subject: [mail <at> ambrevar.xyz: Re: Golang programs keeping references [gnu: go:
 Update default to 1.11.]]
Date: Sat, 15 Dec 2018 14:54:57 -0500
[Message part 1 (text/plain, inline)]
----- Forwarded message from Pierre Neidhardt <mail <at> ambrevar.xyz> -----

Date: Sun, 09 Dec 2018 19:46:03 +0100
From: Pierre Neidhardt <mail <at> ambrevar.xyz>
To: Leo Famulari <leo <at> famulari.name>
Cc: guix-devel <at> gnu.org
Subject: Re: Golang programs keeping references [gnu: go: Update default to 1.11.]
User-agent: mu4e 1.0; emacs 26.1

I've investigated the possible solutions to avoid including the paths into the
binaries.

I've found this:

https://github.com/golang/go/issues/16860

It's still unresolved and only planned for Go 1.13.

In the meantime, I've played with the -trimpath option to see if I could get
something out of it.

I've added this to Go's (build) function:

--8<---------------cut here---------------start------------->8---
              "-asmflags=all=-trimpath=/gnu/store"
              "-gcflags=all=-trimpath=/gnu/store"
--8<---------------cut here---------------end--------------->8---

The resulting binary is indeed trimmed, but that's not enough: it seems that
Guix detects the remaining part of the path as a store item and includes it in
the list of requisites.  Is this really how Guix works?  It does not need the
full path?

Go supports only one call to -trimpath, so we can't even set this to the Go
inputs.

Regarding Boyer-Moore over the binary, we would have to apply the changes for
all recursive Go libraries.  Now is there a reliable way to detect what's a Go
library and what is not?  We don't want to patch non-Go libraries, right?
(Let's not forget that there is CGo...)

If we can't think of a way to detect a Go library, Boyer-Moore does not seem to
be a solution either.  And we might have to wait for Go 1.13...

-- 
Pierre Neidhardt
https://ambrevar.xyz/



----- End forwarded message -----
[signature.asc (application/pgp-signature, inline)]

Merged 33620 33741. Request was from Leo Famulari <leo <at> famulari.name> to control <at> debbugs.gnu.org. (Sun, 16 Dec 2018 21:27:01 GMT) Full text and rfc822 format available.

Reply sent to Leo Famulari <leo <at> famulari.name>:
You have taken responsibility. (Thu, 14 Mar 2019 19:45:01 GMT) Full text and rfc822 format available.

Notification sent to Leo Famulari <leo <at> famulari.name>:
bug acknowledged by developer. (Thu, 14 Mar 2019 19:45:02 GMT) Full text and rfc822 format available.

Message #15 received at 33620-done <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: 33620-done <at> debbugs.gnu.org
Subject: Re: [mail <at> ambrevar.xyz: Re: Golang programs keeping references [gnu:
 go: Update default to 1.11.]]
Date: Thu, 14 Mar 2019 15:44:15 -0400
[Message part 1 (text/plain, inline)]
The issue of Go packages keeping references to all their Go-language is
inputs is resolved with commit e3900a4d64e4bf6f426809d6bff058e5a2ae9bc8.

This commit basically avoids bringing the store paths into the build
environment at all by symlinking them into a union directory in the
TMPDIR. This is a bit of a hack but it's actually more idiomatic in Go
to have all the inputs in a single directory ("workspace" in Go). The
previous technique of passing a list of directories in the GOPATH
variable is officially supported but is definitely a 2nd-class technique
in practice.

----- Forwarded message from Pierre Neidhardt <mail <at> ambrevar.xyz> -----
> I've added this to Go's (build) function:
> 
> --8<---------------cut here---------------start------------->8---
>               "-asmflags=all=-trimpath=/gnu/store"
>               "-gcflags=all=-trimpath=/gnu/store"
> --8<---------------cut here---------------end--------------->8---
> 
> The resulting binary is indeed trimmed, but that's not enough: it seems that
> Guix detects the remaining part of the path as a store item and includes it in
> the list of requisites.  Is this really how Guix works?  It does not need the
> full path?

Not having read the reference scanner code carefully, I expect that it
ignores the '/gnu/store/' since this path is actually configurable.

> Regarding Boyer-Moore over the binary, we would have to apply the changes for
> all recursive Go libraries.  Now is there a reliable way to detect what's a Go
> library and what is not?  We don't want to patch non-Go libraries, right?
> (Let's not forget that there is CGo...)

In (guix build-system go), I'd like to construct of list of inputs that
use the go-build-system and pass this list to the biuld side. This would
be useful for other things, but could also be used to detect which
inputs are Go libraries.

> If we can't think of a way to detect a Go library, Boyer-Moore does not seem to
> be a solution either.  And we might have to wait for Go 1.13...

Waiting for upstream is always the easiest way :)

But it would still be nice to have Boyer-Moore available in (guix build
utils) for whenever we want to use it. Or even in Guile itself...
[signature.asc (application/pgp-signature, inline)]

Reply sent to Leo Famulari <leo <at> famulari.name>:
You have taken responsibility. (Thu, 14 Mar 2019 19:45:02 GMT) Full text and rfc822 format available.

Notification sent to Pierre Neidhardt <mail <at> ambrevar.xyz>:
bug acknowledged by developer. (Thu, 14 Mar 2019 19:45:02 GMT) Full text and rfc822 format available.

bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Fri, 12 Apr 2019 11:24:04 GMT) Full text and rfc822 format available.

This bug report was last modified 4 years and 342 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.