GNU bug report logs - #25235
Wrapped python programs get native-inputs in PYTHONPATH

Previous Next

Package: guix;

Reported by: Marius Bakke <mbakke <at> fastmail.com>

Date: Mon, 19 Dec 2016 23:30:02 UTC

Severity: normal

Fix blocked by 60847: [PATCH] Enable cross-compilation for the pyproject-build-system.

To reply to this bug, email your comments to 25235 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-guix <at> gnu.org:
bug#25235; Package guix. (Mon, 19 Dec 2016 23:30:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Marius Bakke <mbakke <at> fastmail.com>:
New bug report received and forwarded. Copy sent to bug-guix <at> gnu.org. (Mon, 19 Dec 2016 23:30:03 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Marius Bakke <mbakke <at> fastmail.com>
To: bug-guix <at> gnu.org
Subject: Wrapped python programs get native-inputs in PYTHONPATH
Date: Tue, 20 Dec 2016 00:28:58 +0100
[Message part 1 (text/plain, inline)]
The 'wrap' phase of 'python-build-system' adds native-inputs to
PYTHONPATH of the wrapped programs. This causes unnecessary runtime
dependencies and may create problems when cross-compiling.

Given that all python dependencies are now supposed to be propagated,
perhaps we can simply remove the wrap phase? It only touches PYTHONPATH.
[signature.asc (application/pgp-signature, inline)]

Information forwarded to bug-guix <at> gnu.org:
bug#25235; Package guix. (Tue, 20 Dec 2016 14:02:01 GMT) Full text and rfc822 format available.

Message #8 received at 25235 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Marius Bakke <mbakke <at> fastmail.com>
Cc: 25235 <at> debbugs.gnu.org
Subject: Re: bug#25235: Wrapped python programs get native-inputs in PYTHONPATH
Date: Tue, 20 Dec 2016 15:00:50 +0100
Marius Bakke <mbakke <at> fastmail.com> skribis:

> The 'wrap' phase of 'python-build-system' adds native-inputs to
> PYTHONPATH of the wrapped programs. This causes unnecessary runtime
> dependencies and may create problems when cross-compiling.
>
> Given that all python dependencies are now supposed to be propagated,
> perhaps we can simply remove the wrap phase? It only touches PYTHONPATH.

We cannot simply remove it: it is meant for packages that provide
standalone programs, such as ‘certbot’.  For these, it automatically
wraps bin/* and we don’t even have to have propagated inputs, which is
pretty cool.

However, you are right that native inputs shouldn’t be there.  Probably
we need to add some filtering in the ‘wrap’ phase?

Ludo’.




Information forwarded to bug-guix <at> gnu.org:
bug#25235; Package guix. (Mon, 26 Dec 2016 18:27:02 GMT) Full text and rfc822 format available.

Message #11 received at 25235 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Marius Bakke <mbakke <at> fastmail.com>, 25235 <at> debbugs.gnu.org
Subject: Re: bug#25235: Wrapped python programs get native-inputs in PYTHONPATH
Date: Mon, 26 Dec 2016 13:26:08 -0500
On Tue, Dec 20, 2016 at 03:00:50PM +0100, Ludovic Courtès wrote:
> Marius Bakke <mbakke <at> fastmail.com> skribis:
> 
> > The 'wrap' phase of 'python-build-system' adds native-inputs to
> > PYTHONPATH of the wrapped programs. This causes unnecessary runtime
> > dependencies and may create problems when cross-compiling.
> >
> > Given that all python dependencies are now supposed to be propagated,
> > perhaps we can simply remove the wrap phase? It only touches PYTHONPATH.
> 
> We cannot simply remove it: it is meant for packages that provide
> standalone programs, such as ‘certbot’.  For these, it automatically
> wraps bin/* and we don’t even have to have propagated inputs, which is
> pretty cool.
> 
> However, you are right that native inputs shouldn’t be there.  Probably
> we need to add some filtering in the ‘wrap’ phase?

While looking at the beets package definition with fresh eyes, I realize
that it will surely break once this change is made. So it will make a
good test case for this improvement :)




Information forwarded to bug-guix <at> gnu.org:
bug#25235; Package guix. (Wed, 29 Mar 2017 11:28:01 GMT) Full text and rfc822 format available.

Message #14 received at 25235 <at> debbugs.gnu.org (full text, mbox):

From: Arun Isaac <arunisaac <at> systemreboot.net>
To: 25235 <at> debbugs.gnu.org
Subject: Re: bug#25235: Wrapped python programs get native-inputs in PYTHONPATH
Date: Wed, 29 Mar 2017 16:57:12 +0530
I'm working on fixing this bug. I have modified
guix/build/python-build-system.scm for the same. In particular, I have
added #:use-module (guix packages) because I need the functions
`package-name' and `package-transitive-target-inputs'. But, when I try
building any python-build-system package with something like
"./pre-inst-env guix build scons", I get a ("no code for module" (guix
packages)) error. What am I missing?

Full backtrace follows:

Backtrace:
In ice-9/eval.scm:
 432: 19 [eval # ()]
In system/base/compile.scm:
 152: 18 [compile-file "/gnu/store/38rxls2y8v8zjva7xq7zv99rv6c81bq3-module-import/guix/build/python-build-system.scm" ...]
  43: 17 [call-once #<procedure ae84c0 at system/base/compile.scm:56:5 ()>]
In ice-9/boot-9.scm:
 174: 16 [with-throw-handler #t ...]
In system/base/compile.scm:
  59: 15 [#<procedure ae8480 at system/base/compile.scm:58:9 ()>]
 155: 14 [#<procedure ae8500 at system/base/compile.scm:153:8 (port)> #<closed: file 0>]
 218: 13 [read-and-compile #<input: /gnu/store/38rxls2y8v8zjva7xq7zv99rv6c81bq3-module-import/guix/build/python-build-system.scm 6> ...]
 234: 12 [lp () #f #<module (#{ g152}#) ae7a20>]
 182: 11 [lp (#<procedure compile-tree-il (x e opts)>) (define-module # # ...) ...]
In ice-9/boot-9.scm:
2404: 10 [save-module-excursion #<procedure c967e0 at language/scheme/compile-tree-il.scm:29:3 ()>]
In language/scheme/compile-tree-il.scm:
  31: 9 [#<procedure c967e0 at language/scheme/compile-tree-il.scm:29:3 ()>]
In ice-9/psyntax.scm:
1106: 8 [expand-top-sequence ((define-module # # # ...)) () ((top)) ...]
 989: 7 [scan ((define-module # # # ...)) () ((top)) ...]
 279: 6 [scan ((#(syntax-object let # ...) (#) (# #) ...)) () ...]
In ice-9/eval.scm:
 411: 5 [eval # ()]
In ice-9/boot-9.scm:
2954: 4 [define-module* (guix build python-build-system) #:filename ...]
2929: 3 [resolve-imports ((# # gnu:) (#) (#) (#) ...)]
2870: 2 [resolve-interface (guix packages) #:select ...]
In unknown file:
   ?: 1 [scm-error misc-error #f "~A ~S" ("no code for module" (guix packages)) #f]
In ice-9/boot-9.scm:
 109: 0 [#<procedure ae8440 at ice-9/boot-9.scm:100:6 (thrown-k . args)> misc-error ...]

ice-9/boot-9.scm:109:20: In procedure #<procedure ae8440 at ice-9/boot-9.scm:100:6 (thrown-k . args)>:
ice-9/boot-9.scm:109:20: no code for module (guix packages)
builder for `/gnu/store/lsrxzd1hdsgpb9zhrdv02dsxihpmxmww-module-import-compiled.drv' failed with exit code 1
@ build-failed /gnu/store/lsrxzd1hdsgpb9zhrdv02dsxihpmxmww-module-import-compiled.drv - 1 builder for `/gnu/store/lsrxzd1hdsgpb9zhrdv02dsxihpmxmww-module-import-compiled.drv' failed with exit code 1
cannot build derivation `/gnu/store/pm2r6cbcfij8d7nzdxkzqx1dq880mhg8-scons-2.3.4.drv': 1 dependencies couldn't be built
guix build: error: build failed: build of `/gnu/store/pm2r6cbcfij8d7nzdxkzqx1dq880mhg8-scons-2.3.4.drv' failed




Information forwarded to bug-guix <at> gnu.org:
bug#25235; Package guix. (Thu, 30 Mar 2017 15:31:02 GMT) Full text and rfc822 format available.

Message #17 received at 25235 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Arun Isaac <arunisaac <at> systemreboot.net>
Cc: 25235 <at> debbugs.gnu.org
Subject: Re: bug#25235: Wrapped python programs get native-inputs in PYTHONPATH
Date: Thu, 30 Mar 2017 17:30:25 +0200
[Message part 1 (text/plain, inline)]
Hello,

Arun Isaac <arunisaac <at> systemreboot.net> skribis:

> I'm working on fixing this bug. I have modified
> guix/build/python-build-system.scm for the same. In particular, I have
> added #:use-module (guix packages) because I need the functions
> `package-name' and `package-transitive-target-inputs'. But, when I try
> building any python-build-system package with something like
> "./pre-inst-env guix build scons", I get a ("no code for module" (guix
> packages)) error. What am I missing?
>
> Full backtrace follows:

[...]

> 2954: 4 [define-module* (guix build python-build-system) #:filename ...]
> 2929: 3 [resolve-imports ((# # gnu:) (#) (#) (#) ...)]
> 2870: 2 [resolve-interface (guix packages) #:select ...]
> In unknown file:
>    ?: 1 [scm-error misc-error #f "~A ~S" ("no code for module" (guix packages)) #f]

“Build-side” modules, which typically live in (guix build …), should not
depend on “host-side” modules such as (guix packages).  That’s because
if we did that, we’d effectively end up importing all of Guix on the
build side, but then we’d also have to serialize data structures such as
packages to pass them from one side to the other.  (I hope this makes
sense to you, but if it doesn’t maybe the intro of
<https://gnu.org/s/guix/manual/html_node/G_002dExpressions.html> can
shed some light.)

So in short, we cannot use ‘package-name’ and
‘package-transitive-target-inputs’ in this module.

(Time passes…)

I wasn’t sure how to fix this bug myself so I gave it a try and ended up
with the patch below, but I haven’t tested in detail.  (You’ll notice
(guix build-system python) is hard to work with because it doesn’t use
gexps yet.)

How does it look?

Thanks,
Ludo’.

[Message part 2 (text/x-patch, inline)]
diff --git a/guix/build-system/python.scm b/guix/build-system/python.scm
index 17173f121..a05fd5a79 100644
--- a/guix/build-system/python.scm
+++ b/guix/build-system/python.scm
@@ -153,7 +153,7 @@ pre-defined variants."
                 #:rest arguments)
   "Return a bag for NAME."
   (define private-keywords
-    '(#:source #:target #:python #:inputs #:native-inputs))
+    '(#:source #:target #:python #:inputs))
 
   (and (not target)                               ;XXX: no cross-compilation
        (bag
@@ -174,6 +174,9 @@ pre-defined variants."
 
 (define* (python-build store name inputs
                        #:key
+                       (native-inputs '())
+                       ;; TODO: Something like this:
+                       ;; (disallowed-references native-inputs)
                        (tests? #t)
                        (test-target "test")
                        (use-setuptools? #t)
@@ -189,17 +192,24 @@ pre-defined variants."
                                   (guix build utils))))
   "Build SOURCE using PYTHON, and with INPUTS.  This assumes that SOURCE
 provides a 'setup.py' file as its build system."
+  (define canonicalize-reference
+    (match-lambda
+      (((? derivation? source))
+       (derivation->output-path source))
+      (((? package? package))
+       (derivation->output-path
+        (package-derivation store package system)))
+      ((source)
+       source)
+      (source
+       source)))
+
   (define builder
     `(begin
        (use-modules ,@modules)
        (python-build #:name ,name
-                     #:source ,(match (assoc-ref inputs "source")
-                                 (((? derivation? source))
-                                  (derivation->output-path source))
-                                 ((source)
-                                  source)
-                                 (source
-                                  source))
+                     #:source ,(canonicalize-reference
+                                (assoc-ref inputs "source"))
                      #:configure-flags ,configure-flags
                      #:system ,system
                      #:test-target ,test-target
@@ -209,7 +219,17 @@ provides a 'setup.py' file as its build system."
                      #:outputs %outputs
                      #:search-paths ',(map search-path-specification->sexp
                                            search-paths)
-                     #:inputs %build-inputs)))
+                     #:inputs %build-inputs
+
+                     ;; We call them "native inputs" but there's no
+                     ;; cross-compilation here, so that really means
+                     ;; "build-time-only" inputs, things should not be
+                     ;; run-time dependencies.
+                     #:native-inputs ',(map (match-lambda
+                                              ((name . rest)
+                                               `(,name
+                                                 . ,(canonicalize-reference rest))))
+                                            native-inputs))))
 
   (define guile-for-build
     (match guile
@@ -222,6 +242,8 @@ provides a 'setup.py' file as its build system."
 
   (build-expression->derivation store name builder
                                 #:inputs inputs
+                                ;; TODO:
+                                ;; #:disallowed-references disallowed-references
                                 #:system system
                                 #:modules imported-modules
                                 #:outputs outputs
diff --git a/guix/build/python-build-system.scm b/guix/build/python-build-system.scm
index dd07986b9..1ca26104b 100644
--- a/guix/build/python-build-system.scm
+++ b/guix/build/python-build-system.scm
@@ -1,5 +1,5 @@
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2013, 2015, 2016 Ludovic Courtès <ludo <at> gnu.org>
+;;; Copyright © 2013, 2015, 2016, 2017 Ludovic Courtès <ludo <at> gnu.org>
 ;;; Copyright © 2013 Andreas Enge <andreas <at> enge.fr>
 ;;; Copyright © 2013 Nikita Karetnikov <nikita <at> karetnikov.org>
 ;;; Copyright © 2015 Mark H Weaver <mhw <at> netris.org>
@@ -184,7 +184,7 @@ when running checks after installing the package."
                          configure-flags)))
     (call-setuppy "install" params use-setuptools?)))
 
-(define* (wrap #:key inputs outputs #:allow-other-keys)
+(define* (wrap #:key native-inputs inputs outputs #:allow-other-keys)
   (define (list-of-files dir)
     (map (cut string-append dir "/" <>)
          (or (scandir dir (lambda (f)
@@ -199,14 +199,27 @@ when running checks after installing the package."
                         (string-append dir "/sbin"))))
                 outputs))
 
+  (define build-time-inputs
+    ;; Built-time-only dependencies.
+    (match native-inputs
+      (((names . directories) ...)
+       directories)))
+
+  (define (build-time-dependency? item)
+    (any (cut string-prefix? <> item)
+         build-time-inputs))
+
+  ;; Wrap binaries such that PYTHONPATH is set appropriately, but remove
+  ;; build-time-only dependencies (aka. #:native-inputs) from the search path.
   (let* ((out  (assoc-ref outputs "out"))
          (python (assoc-ref inputs "python"))
+         (path (search-path-as-string->list
+                (or (getenv "PYTHONPATH") "")))
          (var `("PYTHONPATH" prefix
                 ,(cons (string-append out "/lib/python"
                                       (get-python-version python)
                                       "/site-packages")
-                       (search-path-as-string->list
-                        (or (getenv "PYTHONPATH") ""))))))
+                       (remove build-time-dependency? path)))))
     (for-each (lambda (dir)
                 (let ((files (list-of-files dir)))
                   (for-each (cut wrap-program <> var)


Information forwarded to bug-guix <at> gnu.org:
bug#25235; Package guix. (Thu, 30 Mar 2017 19:09:01 GMT) Full text and rfc822 format available.

Message #20 received at 25235 <at> debbugs.gnu.org (full text, mbox):

From: Arun Isaac <arunisaac <at> systemreboot.net>
To: 25235 <at> debbugs.gnu.org
Subject: Re: bug#25235: Wrapped python programs get native-inputs in PYTHONPATH
Date: Fri, 31 Mar 2017 00:37:53 +0530
> “Build-side” modules, which typically live in (guix build …), should not
> depend on “host-side” modules such as (guix packages).  That’s because
> if we did that, we’d effectively end up importing all of Guix on the
> build side, but then we’d also have to serialize data structures such as
> packages to pass them from one side to the other.  (I hope this makes
> sense to you, but if it doesn’t maybe the intro of
> <https://gnu.org/s/guix/manual/html_node/G_002dExpressions.html> can
> shed some light.)

It makes some sense, but I'll read up more.

> So in short, we cannot use ‘package-name’ and
> ‘package-transitive-target-inputs’ in this module.

Ok.

> (Time passes…)
>
> I wasn’t sure how to fix this bug myself so I gave it a try and ended up
> with the patch below, but I haven’t tested in detail.  (You’ll notice
> (guix build-system python) is hard to work with because it doesn’t use
> gexps yet.)
>
> How does it look?

I'm travelling now. I'll get back on Monday and study this patch.




Information forwarded to bug-guix <at> gnu.org:
bug#25235; Package guix. (Tue, 04 Apr 2017 17:03:02 GMT) Full text and rfc822 format available.

Message #23 received at 25235 <at> debbugs.gnu.org (full text, mbox):

From: Arun Isaac <arunisaac <at> systemreboot.net>
To: 25235 <at> debbugs.gnu.org
Subject: Re: bug#25235: Wrapped python programs get native-inputs in PYTHONPATH
Date: Tue, 04 Apr 2017 22:32:24 +0530
> How does it look?

I tried your patch. But native-inputs are still there in the wrapped
PYTHONPATH. I printed out the results from various steps from the wrap
phase. They are as follows:

`build-time-inputs' is

("/gnu/store/5wbi6vsi12bmpzygwhys3c0cc6sjdw40-python-beautifulsoup4-4.5.3"
"/gnu/store/kc5gl1j85y9ispjsgbg5f5c6jdy1k32s-python-flask-0.11.1"
"/gnu/store/3ib43rpca6qzwfcav8s9ivm3dikpxq87-python-mock-1.0.1"
"/gnu/store/6q4hszis7z6x44ig2zlwjwlcjy3vvfmp-python-mpd2-0.5.5"
"/gnu/store/5wx6yffx6dppd2dr91qfi1jlpqlp9f5m-python-nose-1.3.7"
"/gnu/store/kqjfgfwzm7sq43fnng1w23mjyr2xrd75-python-pathlib-1.0.1"
"/gnu/store/b9yn8bfd43pvrp2wyinf6rf2qw113r5p-python-pyxdg-0.25"
"/gnu/store/g4vfl4wy81y14fjhlpmgqh0i01cg37mb-python-pylast-1.6.0"
"/gnu/store/nim5r75b1kzawln3a1bgag2f12vdmb8x-python-rarfile-2.8"
"/gnu/store/wbaaacm1m7bzbhi3z3scwbz2p4pf2ybg-python-responses-0.5.1")

`path' is

("/gnu/store/5wbi6vsi12bmpzygwhys3c0cc6sjdw40-python-beautifulsoup4-4.5.3/lib/python3.5/site-packages"
"/gnu/store/r3f04sda4gh5vbsa7zb5x0ygf326cp75-python-flask-0.11.1/lib/python3.5/site-packages"
"/gnu/store/3ib43rpca6qzwfcav8s9ivm3dikpxq87-python-mock-1.0.1/lib/python3.5/site-packages"
"/gnu/store/5mkb6im3dijkw4f5146mjz4c49qgcklq-python-mpd2-0.5.5/lib/python3.5/site-packages"
"/gnu/store/9z450rcsza3x448ky83fkd5636ck9aqd-python-nose-1.3.7/lib/python3.5/site-packages"
"/gnu/store/kqjfgfwzm7sq43fnng1w23mjyr2xrd75-python-pathlib-1.0.1/lib/python3.5/site-packages"
"/gnu/store/b9yn8bfd43pvrp2wyinf6rf2qw113r5p-python-pyxdg-0.25/lib/python3.5/site-packages"
"/gnu/store/g4vfl4wy81y14fjhlpmgqh0i01cg37mb-python-pylast-1.6.0/lib/python3.5/site-packages"
"/gnu/store/nim5r75b1kzawln3a1bgag2f12vdmb8x-python-rarfile-2.8/lib/python3.5/site-packages"
"/gnu/store/wbaaacm1m7bzbhi3z3scwbz2p4pf2ybg-python-responses-0.5.1/lib/python3.5/site-packages"
"/gnu/store/m6ir6mj7laxcwsjrz0lvxm7q04w0anhp-python-discogs-client-2.2.1/lib/python3.5/site-packages"
"/gnu/store/v3xiyw0cj08qmpn08jaiznd8ak3nvdga-python-jellyfish-0.5.6/lib/python3.5/site-packages"
"/gnu/store/pwxvvi24950x6rgvl4f46z89sbpa2b7a-python-munkres-1.0.8/lib/python3.5/site-packages"
"/gnu/store/nws3867b5q78ri461wr3rdwgqzasvv2a-python-musicbrainzngs-0.6/lib/python3.5/site-packages"
"/gnu/store/wsw2nr30nhz29dww1hfjb6lgqn2ckrgy-python-mutagen-1.36/lib/python3.5/site-packages"
"/gnu/store/n7fh268b1g1g87s4djvm672p2xmwkd5r-python-pyyaml-3.12/lib/python3.5/site-packages"
"/gnu/store/y3ncl6ia04vqdkhnhgzb3iabcsczj0zb-python-unidecode-0.04.20/lib/python3.5/site-packages"
"/gnu/store/alk9r3rir93pjmv8im20f8xrvv90219z-python-3.5.2/lib/python3.5/site-packages"
"/gnu/store/z2mmzglkykws7hc9k2sr40fidiv62cpa-python-werkzeug-0.11.15/lib/python3.5/site-packages"
"/gnu/store/5w60y9zyjskv179sf4pmq2kzpff887dv-python-click-6.7/lib/python3.5/site-packages"
"/gnu/store/f4qqgaql0f95alxrz9r52n3m5ahjrc1s-python-jinja2-2.8/lib/python3.5/site-packages"
"/gnu/store/n0l3szawn661n2rjk6nnbajzzc02pmfj-python-itsdangerous-0.24/lib/python3.5/site-packages"
"/gnu/store/rpnfm1v7vnwxl3wpbadz2bz25s2x1bbg-python-six-1.10.0/lib/python3.5/site-packages"
"/gnu/store/64ahrdmjmmm8id5327i5kqjj8ppvi2kl-python-cookies-2.2.1/lib/python3.5/site-packages"
"/gnu/store/2iyrxbr6k1qdcrhfpv3cz1nh1znwnga5-python-requests-2.13.0/lib/python3.5/site-packages"
"/gnu/store/0c28plwcf33lnibf27h8ijig467pphiw-python-oauthlib-1.0.3/lib/python3.5/site-packages"
"/gnu/store/m48bcrmrzrcb79p6dhaq0diwr3aacfq3-python-markupsafe-0.23/lib/python3.5/site-packages")

(remove build-time-dependency? path) is

("/gnu/store/r3f04sda4gh5vbsa7zb5x0ygf326cp75-python-flask-0.11.1/lib/python3.5/site-packages"
"/gnu/store/5mkb6im3dijkw4f5146mjz4c49qgcklq-python-mpd2-0.5.5/lib/python3.5/site-packages"
"/gnu/store/9z450rcsza3x448ky83fkd5636ck9aqd-python-nose-1.3.7/lib/python3.5/site-packages"
"/gnu/store/m6ir6mj7laxcwsjrz0lvxm7q04w0anhp-python-discogs-client-2.2.1/lib/python3.5/site-packages"
"/gnu/store/v3xiyw0cj08qmpn08jaiznd8ak3nvdga-python-jellyfish-0.5.6/lib/python3.5/site-packages"
"/gnu/store/pwxvvi24950x6rgvl4f46z89sbpa2b7a-python-munkres-1.0.8/lib/python3.5/site-packages"
"/gnu/store/nws3867b5q78ri461wr3rdwgqzasvv2a-python-musicbrainzngs-0.6/lib/python3.5/site-packages"
"/gnu/store/wsw2nr30nhz29dww1hfjb6lgqn2ckrgy-python-mutagen-1.36/lib/python3.5/site-packages"
"/gnu/store/n7fh268b1g1g87s4djvm672p2xmwkd5r-python-pyyaml-3.12/lib/python3.5/site-packages"
"/gnu/store/y3ncl6ia04vqdkhnhgzb3iabcsczj0zb-python-unidecode-0.04.20/lib/python3.5/site-packages"
"/gnu/store/alk9r3rir93pjmv8im20f8xrvv90219z-python-3.5.2/lib/python3.5/site-packages"
"/gnu/store/z2mmzglkykws7hc9k2sr40fidiv62cpa-python-werkzeug-0.11.15/lib/python3.5/site-packages"
"/gnu/store/5w60y9zyjskv179sf4pmq2kzpff887dv-python-click-6.7/lib/python3.5/site-packages"
"/gnu/store/f4qqgaql0f95alxrz9r52n3m5ahjrc1s-python-jinja2-2.8/lib/python3.5/site-packages"
"/gnu/store/n0l3szawn661n2rjk6nnbajzzc02pmfj-python-itsdangerous-0.24/lib/python3.5/site-packages"
"/gnu/store/rpnfm1v7vnwxl3wpbadz2bz25s2x1bbg-python-six-1.10.0/lib/python3.5/site-packages"
"/gnu/store/64ahrdmjmmm8id5327i5kqjj8ppvi2kl-python-cookies-2.2.1/lib/python3.5/site-packages"
"/gnu/store/2iyrxbr6k1qdcrhfpv3cz1nh1znwnga5-python-requests-2.13.0/lib/python3.5/site-packages"
"/gnu/store/0c28plwcf33lnibf27h8ijig467pphiw-python-oauthlib-1.0.3/lib/python3.5/site-packages"
"/gnu/store/m48bcrmrzrcb79p6dhaq0diwr3aacfq3-python-markupsafe-0.23/lib/python3.5/site-packages")

Somehow, the paths in `build-time-inputs' are not the same as those in
`path'. Notice the different hashes. Hence `remove' does not succeed in
filtering them out.

> +                       ;; TODO: Something like this:
> +                       ;; (disallowed-references native-inputs)

What is this part for?




Information forwarded to bug-guix <at> gnu.org:
bug#25235; Package guix. (Thu, 07 Feb 2019 13:23:01 GMT) Full text and rfc822 format available.

Message #26 received at 25235 <at> debbugs.gnu.org (full text, mbox):

From: Ricardo Wurmus <rekado <at> elephly.net>
To: Arun Isaac <arunisaac <at> systemreboot.net>
Cc: 25235 <at> debbugs.gnu.org
Subject: Re: bug#25235: Wrapped python programs get native-inputs in PYTHONPATH
Date: Thu, 07 Feb 2019 14:22:07 +0100
Arun Isaac <arunisaac <at> systemreboot.net> writes:

>> +                       ;; TODO: Something like this:
>> +                       ;; (disallowed-references native-inputs)
>
> What is this part for?

This is to ensure that no references to any native input are retained in
the output.  (Think of it as an assertion that is checked at the end of
the build.)

~~ Ricardo





Information forwarded to bug-guix <at> gnu.org:
bug#25235; Package guix. (Mon, 06 Apr 2020 15:57:02 GMT) Full text and rfc822 format available.

Message #29 received at 25235 <at> debbugs.gnu.org (full text, mbox):

From: Ricardo Wurmus via web <issues.guix.gnu.org <at> elephly.net>
To: 25235 <at> debbugs.gnu.org
Date: Mon,  6 Apr 2020 16:49:07 +0200
Have you been able to figure out why some of the items differ in the two lists?





Information forwarded to bug-guix <at> gnu.org:
bug#25235; Package guix. (Thu, 09 Apr 2020 00:04:01 GMT) Full text and rfc822 format available.

Message #32 received at 25235 <at> debbugs.gnu.org (full text, mbox):

From: Arun Isaac <arunisaac <at> systemreboot.net>
To: Ricardo Wurmus <rekado <at> elephly.net>
Cc: 25235 <at> debbugs.gnu.org
Subject: Re: bug#25235: (no subject)
Date: Thu, 09 Apr 2020 05:32:39 +0530
[Message part 1 (text/plain, inline)]
> Have you been able to figure out why some of the items differ in the two lists?

Hi! Are you asking me? I haven't worked on this in a very long
time. But I'll give it another shot now and get back to you.
[signature.asc (application/pgp-signature, inline)]

Information forwarded to bug-guix <at> gnu.org:
bug#25235; Package guix. (Tue, 21 Apr 2020 02:16:01 GMT) Full text and rfc822 format available.

Message #35 received at 25235 <at> debbugs.gnu.org (full text, mbox):

From: Arun Isaac <arunisaac <at> systemreboot.net>
To: Ricardo Wurmus <rekado <at> elephly.net>
Cc: 25235 <at> debbugs.gnu.org
Subject: Re: bug#25235: Wrapped python programs get native-inputs in PYTHONPATH
Date: Tue, 21 Apr 2020 07:45:32 +0530
[Message part 1 (text/plain, inline)]
Hi,

> Have you been able to figure out why some of the items differ in the
> two lists?

The two lists are the same if `guix build' is invoked with the
`--no-grafts' flag. I don't understand grafts very well and am not sure
how that can be fixed. Any ideas?

See different output below with and without the `--no-grafts' flag.

$ guix build python-mock

--8<---------------cut here---------------start------------->8---
`build-time-inputs' is
(/gnu/store/dhs3v8rfzazq9cb37pqs7fdm5hd7zmf0-python-unittest2-1.1.0)

`path' is
(/gnu/store/v70qcgga1r2m6scnard8s61dqh859mn2-python-unittest2-1.1.0/lib/python3.7/site-packages
/gnu/store/w408dkc1lynq5kqi2cxkvikalp6w35qw-python-pbr-minimal-3.0.1/lib/python3.7/site-packages
/gnu/store/bwqidwykphzv10n454kxv3hfippq8qrx-python-six-1.12.0/lib/python3.7/site-packages
/gnu/store/78w7y0lxar70j512iqw8x3nimzj10yga-python-3.7.4/lib/python3.7/site-packages
/gnu/store/k4a9433nghpjhs101cm9v8k64bmd8iqf-python-traceback2-1.4.0/lib/python3.7/site-packages
/gnu/store/3nl9hk7f59vsxgr7xmnqgc15y9c3rwpg-python-linecache2-1.0.0/lib/python3.7/site-packages)

`(remove build-time-dependency? path)' is
(/gnu/store/v70qcgga1r2m6scnard8s61dqh859mn2-python-unittest2-1.1.0/lib/python3.7/site-packages
/gnu/store/w408dkc1lynq5kqi2cxkvikalp6w35qw-python-pbr-minimal-3.0.1/lib/python3.7/site-packages
/gnu/store/bwqidwykphzv10n454kxv3hfippq8qrx-python-six-1.12.0/lib/python3.7/site-packages
/gnu/store/78w7y0lxar70j512iqw8x3nimzj10yga-python-3.7.4/lib/python3.7/site-packages
/gnu/store/k4a9433nghpjhs101cm9v8k64bmd8iqf-python-traceback2-1.4.0/lib/python3.7/site-packages
/gnu/store/3nl9hk7f59vsxgr7xmnqgc15y9c3rwpg-python-linecache2-1.0.0/lib/python3.7/site-packages)
--8<---------------cut here---------------end--------------->8---

$ guix build --no-grafts python-mock

--8<---------------cut here---------------start------------->8---
`build-time-inputs' is
(/gnu/store/m01qbb9sgymxh2xqwpnbv58h60yl4pz0-python-unittest2-1.1.0)

`path' is
(/gnu/store/m01qbb9sgymxh2xqwpnbv58h60yl4pz0-python-unittest2-1.1.0/lib/python3.7/site-packages
/gnu/store/9k5hyd9d7b3r4yzadd6phphf12l2mfc8-python-pbr-minimal-3.0.1/lib/python3.7/site-packages
/gnu/store/p8p65yd0pf95yn50iib0jw05p0n4mr9p-python-six-1.12.0/lib/python3.7/site-packages
/gnu/store/78w7y0lxar70j512iqw8x3nimzj10yga-python-3.7.4/lib/python3.7/site-packages
/gnu/store/dvn1fz82ccjg7dnqqmkxhwp720wnmriw-python-traceback2-1.4.0/lib/python3.7/site-packages
/gnu/store/vnbk5jswscv68qasfbhmh7wqlsibyism-python-linecache2-1.0.0/lib/python3.7/site-packages)

`(remove build-time-dependency? path)' is
(/gnu/store/9k5hyd9d7b3r4yzadd6phphf12l2mfc8-python-pbr-minimal-3.0.1/lib/python3.7/site-packages
/gnu/store/p8p65yd0pf95yn50iib0jw05p0n4mr9p-python-six-1.12.0/lib/python3.7/site-packages
/gnu/store/78w7y0lxar70j512iqw8x3nimzj10yga-python-3.7.4/lib/python3.7/site-packages
/gnu/store/dvn1fz82ccjg7dnqqmkxhwp720wnmriw-python-traceback2-1.4.0/lib/python3.7/site-packages
/gnu/store/vnbk5jswscv68qasfbhmh7wqlsibyism-python-linecache2-1.0.0/lib/python3.7/site-packages)
--8<---------------cut here---------------end--------------->8---

Regards,
Arun
[signature.asc (application/pgp-signature, inline)]

Information forwarded to bug-guix <at> gnu.org:
bug#25235; Package guix. (Mon, 16 Jan 2023 21:30:02 GMT) Full text and rfc822 format available.

Message #38 received at 25235 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: 25235 <at> debbugs.gnu.org
Cc: arunisaac <at> systemreboot.net, Maxim Cournoyer <maxim.cournoyer <at> gmail.com>,
 ludo <at> gnu.org, Lars-Dominik Braun <lars <at> 6xq.net>, rekado <at> elephly.net,
 jgart <jgart <at> dismail.de>, marius <at> gnu.org
Subject: [PATCH 0/1] build-system/pyproject: Do not wrap native-inputs.
Date: Mon, 16 Jan 2023 16:29:18 -0500
This fixes a longstanding issue.  The fix is made trivial by the changes made
in #60847 adding cross-compilation support for the pyproject build system.
Additionally, it's based on #60868, which touches the wrap phase to use
wrap-script instead of wrap-program.


Maxim Cournoyer (1):
  build-system/pyproject: Do not wrap native-inputs.

 guix/build/pyproject-build-system.scm | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)


base-commit: 9da36aa1e207ff8a8fb2af8bc9442f9c935dc8db
-- 
2.38.1





Information forwarded to bug-guix <at> gnu.org:
bug#25235; Package guix. (Mon, 16 Jan 2023 21:30:03 GMT) Full text and rfc822 format available.

Message #41 received at 25235 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: 25235 <at> debbugs.gnu.org
Cc: arunisaac <at> systemreboot.net, Maxim Cournoyer <maxim.cournoyer <at> gmail.com>,
 ludo <at> gnu.org, Lars-Dominik Braun <lars <at> 6xq.net>, rekado <at> elephly.net,
 jgart <jgart <at> dismail.de>, marius <at> gnu.org
Subject: [PATCH 1/1] build-system/pyproject: Do not wrap native-inputs.
Date: Mon, 16 Jan 2023 16:29:19 -0500
Fixes <https://issues.guix.gnu.org/25235>.

* guix/build/pyproject-build-system.scm (wrap) [native-inputs]: New argument.
Filter out native inputs from the values in GUIX_PYTHONPATH.

---

 guix/build/pyproject-build-system.scm | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/guix/build/pyproject-build-system.scm b/guix/build/pyproject-build-system.scm
index a66c1fb34a..cd418f7ec9 100644
--- a/guix/build/pyproject-build-system.scm
+++ b/guix/build/pyproject-build-system.scm
@@ -445,7 +445,7 @@ (define* (add-install-to-pythonpath #:key native-inputs outputs
   "A phase that just wraps the 'add-installed-pythonpath' procedure."
   (add-installed-pythonpath native-inputs outputs))
 
-(define* (wrap #:key inputs outputs #:allow-other-keys)
+(define* (wrap #:key native-inputs inputs outputs #:allow-other-keys)
   (define (list-of-files dir)
     (find-files dir (lambda (file stat)
                       (and (eq? 'regular (stat:type stat))
@@ -464,9 +464,17 @@ (define bindirs
   (define %guile (delay (search-input-file inputs "bin/guile")))
   (define (guile) (force %guile))
 
-  (let* ((var `("GUIX_PYTHONPATH" prefix
-                ,(search-path-as-string->list
-                  (or (getenv "GUIX_PYTHONPATH") "")))))
+  ;; Use the same strategy to compute the native-input file names.
+  (define %native-input-dirs (delay (match native-inputs
+                                      (((_ . dir) ...)
+                                       dir))))
+  (define (native-input-dirs) (force %native-input-dirs))
+
+  (let ((var `("GUIX_PYTHONPATH" prefix
+               ,(remove (lambda (x)
+                          (any (cut string-prefix? <> x) (native-input-dirs)))
+                        (search-path-as-string->list
+                         (or (getenv "GUIX_PYTHONPATH") ""))))))
     (for-each (lambda (dir)
                 (let ((files (list-of-files dir)))
                   (for-each (cut wrap-script <> #:guile (guile) var)
-- 
2.38.1





Information forwarded to bug-guix <at> gnu.org:
bug#25235; Package guix. (Tue, 17 Jan 2023 09:29:02 GMT) Full text and rfc822 format available.

Message #44 received at 25235 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Cc: arunisaac <at> systemreboot.net, 25235 <at> debbugs.gnu.org,
 Lars-Dominik Braun <lars <at> 6xq.net>, rekado <at> elephly.net,
 jgart <jgart <at> dismail.de>, marius <at> gnu.org
Subject: Re: [PATCH 1/1] build-system/pyproject: Do not wrap native-inputs.
Date: Tue, 17 Jan 2023 10:28:20 +0100
Hi,

Maxim Cournoyer <maxim.cournoyer <at> gmail.com> skribis:

> Fixes <https://issues.guix.gnu.org/25235>.
>
> * guix/build/pyproject-build-system.scm (wrap) [native-inputs]: New argument.
> Filter out native inputs from the values in GUIX_PYTHONPATH.

Yay, great to see this fixed!

>  guix/build/pyproject-build-system.scm | 16 ++++++++++++----
>  1 file changed, 12 insertions(+), 4 deletions(-)
>
> diff --git a/guix/build/pyproject-build-system.scm b/guix/build/pyproject-build-system.scm
> index a66c1fb34a..cd418f7ec9 100644
> --- a/guix/build/pyproject-build-system.scm
> +++ b/guix/build/pyproject-build-system.scm
> @@ -445,7 +445,7 @@ (define* (add-install-to-pythonpath #:key native-inputs outputs
>    "A phase that just wraps the 'add-installed-pythonpath' procedure."
>    (add-installed-pythonpath native-inputs outputs))
>  
> -(define* (wrap #:key inputs outputs #:allow-other-keys)
> +(define* (wrap #:key native-inputs inputs outputs #:allow-other-keys)
>    (define (list-of-files dir)
>      (find-files dir (lambda (file stat)
>                        (and (eq? 'regular (stat:type stat))
> @@ -464,9 +464,17 @@ (define bindirs
>    (define %guile (delay (search-input-file inputs "bin/guile")))
>    (define (guile) (force %guile))
>  
> -  (let* ((var `("GUIX_PYTHONPATH" prefix
> -                ,(search-path-as-string->list
> -                  (or (getenv "GUIX_PYTHONPATH") "")))))
> +  ;; Use the same strategy to compute the native-input file names.
> +  (define %native-input-dirs (delay (match native-inputs
> +                                      (((_ . dir) ...)
> +                                       dir))))
> +  (define (native-input-dirs) (force %native-input-dirs))
> +
> +  (let ((var `("GUIX_PYTHONPATH" prefix
> +               ,(remove (lambda (x)
> +                          (any (cut string-prefix? <> x) (native-input-dirs)))
> +                        (search-path-as-string->list
> +                         (or (getenv "GUIX_PYTHONPATH") ""))))))

I think there’s no need to delay+thunk things since we’re using them
right away.  How about:

  (define native-input-directories
    (match native-inputs
      …))

  (let ((var `("GUIX_PYTHONPATH" prefix
               ,(remove (lambda (x) … native-input-directories …) …))))
    …)

?

(Same for ‘%guile’ above, but I don’t see it in ‘master’; maybe that’s
from another branch?)

Thanks,
Ludo’.




Information forwarded to bug-guix <at> gnu.org:
bug#25235; Package guix. (Tue, 17 Jan 2023 15:50:01 GMT) Full text and rfc822 format available.

Message #47 received at 25235 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: arunisaac <at> systemreboot.net, 25235 <at> debbugs.gnu.org,
 Lars-Dominik Braun <lars <at> 6xq.net>, rekado <at> elephly.net,
 jgart <jgart <at> dismail.de>, marius <at> gnu.org
Subject: Re: [PATCH 1/1] build-system/pyproject: Do not wrap native-inputs.
Date: Tue, 17 Jan 2023 10:49:00 -0500
Hi Ludovic,

Ludovic Courtès <ludo <at> gnu.org> writes:

> Hi,
>
> Maxim Cournoyer <maxim.cournoyer <at> gmail.com> skribis:
>
>> Fixes <https://issues.guix.gnu.org/25235>.
>>
>> * guix/build/pyproject-build-system.scm (wrap) [native-inputs]: New argument.
>> Filter out native inputs from the values in GUIX_PYTHONPATH.
>
> Yay, great to see this fixed!
>
>>  guix/build/pyproject-build-system.scm | 16 ++++++++++++----
>>  1 file changed, 12 insertions(+), 4 deletions(-)
>>
>> diff --git a/guix/build/pyproject-build-system.scm b/guix/build/pyproject-build-system.scm
>> index a66c1fb34a..cd418f7ec9 100644
>> --- a/guix/build/pyproject-build-system.scm
>> +++ b/guix/build/pyproject-build-system.scm
>> @@ -445,7 +445,7 @@ (define* (add-install-to-pythonpath #:key native-inputs outputs
>>    "A phase that just wraps the 'add-installed-pythonpath' procedure."
>>    (add-installed-pythonpath native-inputs outputs))
>>
>> -(define* (wrap #:key inputs outputs #:allow-other-keys)
>> +(define* (wrap #:key native-inputs inputs outputs #:allow-other-keys)
>>    (define (list-of-files dir)
>>      (find-files dir (lambda (file stat)
>>                        (and (eq? 'regular (stat:type stat))
>> @@ -464,9 +464,17 @@ (define bindirs
>>    (define %guile (delay (search-input-file inputs "bin/guile")))
>>    (define (guile) (force %guile))
>>
>> -  (let* ((var `("GUIX_PYTHONPATH" prefix
>> -                ,(search-path-as-string->list
>> -                  (or (getenv "GUIX_PYTHONPATH") "")))))
>> +  ;; Use the same strategy to compute the native-input file names.
>> +  (define %native-input-dirs (delay (match native-inputs
>> +                                      (((_ . dir) ...)
>> +                                       dir))))
>> +  (define (native-input-dirs) (force %native-input-dirs))
>> +
>> +  (let ((var `("GUIX_PYTHONPATH" prefix
>> +               ,(remove (lambda (x)
>> +                          (any (cut string-prefix? <> x) (native-input-dirs)))
>> +                        (search-path-as-string->list
>> +                         (or (getenv "GUIX_PYTHONPATH") ""))))))
>
> I think there’s no need to delay+thunk things since we’re using them
> right away.  How about:
>
>   (define native-input-directories
>     (match native-inputs
>       …))
>
>   (let ((var `("GUIX_PYTHONPATH" prefix
>                ,(remove (lambda (x) … native-input-directories …) …))))
>     …)
>
> ?

It's a (small) optimization, that was intended to delay things until
they are really needed that was already used for %bash (now %guile) in
the wrap phase of (guix build python-build-system).

The idea is to delay the evaluation of native-input-directories or the
lookup for guile in the inputs until it's really necessary.  For
example, the computations are avoided when there are no "bin" or "sbin"
sub-directories in the outputs.

> (Same for ‘%guile’ above, but I don’t see it in ‘master’; maybe that’s
> from another branch?)

Yes, the change build on both #60847 (Enable cross-compilation for the
pyproject-build-system) and #60868 (Use wrap-script instead of
wrap-program), which should be applied first.  %guile appears in #60868,
and changes that make the above easy (native-inputs always appearing as
native-inputs, even for native builds) are from #60847.

-- 
Thanks,
Maxim




Added blocking bug(s) 60847 Request was from Maxim Cournoyer <maxim.cournoyer <at> gmail.com> to control <at> debbugs.gnu.org. (Tue, 17 Jan 2023 15:50:02 GMT) Full text and rfc822 format available.

Information forwarded to bug-guix <at> gnu.org:
bug#25235; Package guix. (Wed, 18 Jan 2023 09:52:01 GMT) Full text and rfc822 format available.

Message #52 received at 25235 <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Cc: arunisaac <at> systemreboot.net, 25235 <at> debbugs.gnu.org, ludo <at> gnu.org,
 Lars-Dominik Braun <lars <at> 6xq.net>, rekado <at> elephly.net,
 jgart <jgart <at> dismail.de>, marius <at> gnu.org
Subject: Re: bug#25235: [PATCH 1/1] build-system/pyproject: Do not wrap
 native-inputs.
Date: Wed, 18 Jan 2023 11:51:16 +0200
[Message part 1 (text/plain, inline)]
I'm still on my first cup of coffee...

If a python input is in inputs and in native-inputs would it be included
in the wrapper or not? At first glance I'd say no. Does the search path
do deduplication? If it doesn't then it'd work to only remove the first
instance matching from native-input-dirs.

On Mon, Jan 16, 2023 at 04:29:19PM -0500, Maxim Cournoyer wrote:
> Fixes <https://issues.guix.gnu.org/25235>.
> 
> * guix/build/pyproject-build-system.scm (wrap) [native-inputs]: New argument.
> Filter out native inputs from the values in GUIX_PYTHONPATH.
> 
> ---
> 
>  guix/build/pyproject-build-system.scm | 16 ++++++++++++----
>  1 file changed, 12 insertions(+), 4 deletions(-)
> 
> diff --git a/guix/build/pyproject-build-system.scm b/guix/build/pyproject-build-system.scm
> index a66c1fb34a..cd418f7ec9 100644
> --- a/guix/build/pyproject-build-system.scm
> +++ b/guix/build/pyproject-build-system.scm
> @@ -445,7 +445,7 @@ (define* (add-install-to-pythonpath #:key native-inputs outputs
>    "A phase that just wraps the 'add-installed-pythonpath' procedure."
>    (add-installed-pythonpath native-inputs outputs))
>  
> -(define* (wrap #:key inputs outputs #:allow-other-keys)
> +(define* (wrap #:key native-inputs inputs outputs #:allow-other-keys)
>    (define (list-of-files dir)
>      (find-files dir (lambda (file stat)
>                        (and (eq? 'regular (stat:type stat))
> @@ -464,9 +464,17 @@ (define bindirs
>    (define %guile (delay (search-input-file inputs "bin/guile")))
>    (define (guile) (force %guile))
>  
> -  (let* ((var `("GUIX_PYTHONPATH" prefix
> -                ,(search-path-as-string->list
> -                  (or (getenv "GUIX_PYTHONPATH") "")))))
> +  ;; Use the same strategy to compute the native-input file names.
> +  (define %native-input-dirs (delay (match native-inputs
> +                                      (((_ . dir) ...)
> +                                       dir))))
> +  (define (native-input-dirs) (force %native-input-dirs))
> +
> +  (let ((var `("GUIX_PYTHONPATH" prefix
> +               ,(remove (lambda (x)
> +                          (any (cut string-prefix? <> x) (native-input-dirs)))
> +                        (search-path-as-string->list
> +                         (or (getenv "GUIX_PYTHONPATH") ""))))))
>      (for-each (lambda (dir)
>                  (let ((files (list-of-files dir)))
>                    (for-each (cut wrap-script <> #:guile (guile) var)
> -- 
> 2.38.1
> 
> 
> 
> 

-- 
Efraim Flashner   <efraim <at> flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[signature.asc (application/pgp-signature, inline)]

Information forwarded to bug-guix <at> gnu.org:
bug#25235; Package guix. (Wed, 18 Jan 2023 13:24:01 GMT) Full text and rfc822 format available.

Message #55 received at 25235 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: Efraim Flashner <efraim <at> flashner.co.il>
Cc: arunisaac <at> systemreboot.net, 25235 <at> debbugs.gnu.org, ludo <at> gnu.org,
 Lars-Dominik Braun <lars <at> 6xq.net>, rekado <at> elephly.net,
 jgart <jgart <at> dismail.de>, marius <at> gnu.org
Subject: Re: bug#25235: [PATCH 1/1] build-system/pyproject: Do not wrap
 native-inputs.
Date: Wed, 18 Jan 2023 08:22:52 -0500
Hi Efraim,

Efraim Flashner <efraim <at> flashner.co.il> writes:

> I'm still on my first cup of coffee...
>
> If a python input is in inputs and in native-inputs would it be included
> in the wrapper or not? At first glance I'd say no. Does the search path
> do deduplication? If it doesn't then it'd work to only remove the first
> instance matching from native-input-dirs.

It wouldn't be included with the current simple logic, which simply
remove any native input from the items of GUIX_PYTHONPATH.  Good catch!
I'll send a v2 with the refined check.

-- 
Thanks,
Maxim




Information forwarded to bug-guix <at> gnu.org:
bug#25235; Package guix. (Wed, 18 Jan 2023 14:00:02 GMT) Full text and rfc822 format available.

Message #58 received at 25235 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: 25235 <at> debbugs.gnu.org
Cc: arunisaac <at> systemreboot.net, Maxim Cournoyer <maxim.cournoyer <at> gmail.com>,
 ludo <at> gnu.org, Lars-Dominik Braun <lars <at> 6xq.net>, jgart <jgart <at> dismail.de>,
 marius <at> gnu.org
Subject: [PATCH v2 0/1] build-system/pyproject: Do not wrap native-inputs.
Date: Wed, 18 Jan 2023 08:59:06 -0500
This fixes a longstanding issue.  The fix is made trivial by the changes made
in #60847 adding cross-compilation support for the pyproject build system.
Additionally, it's based on #60868, which touches the wrap phase to use
wrap-script instead of wrap-program.

Changes in v2:
- Add missing copyright line
- Rework wrap phase to avoid removing inputs found in both native-inputs and
inputs
- Enclose wrap computations in an 'unless' form and streamline

Maxim Cournoyer (1):
  build-system/pyproject: Do not wrap native-inputs.

 guix/build/pyproject-build-system.scm | 39 ++++++++++++++++-----------
 1 file changed, 24 insertions(+), 15 deletions(-)


base-commit: 9da36aa1e207ff8a8fb2af8bc9442f9c935dc8db
-- 
2.39.1





Information forwarded to bug-guix <at> gnu.org:
bug#25235; Package guix. (Wed, 18 Jan 2023 14:01:02 GMT) Full text and rfc822 format available.

Message #61 received at 25235 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: 25235 <at> debbugs.gnu.org
Cc: arunisaac <at> systemreboot.net, Maxim Cournoyer <maxim.cournoyer <at> gmail.com>,
 ludo <at> gnu.org, Lars-Dominik Braun <lars <at> 6xq.net>, efraim <at> flashner.co.il,
 rekado <at> elephly.net, jgart <jgart <at> dismail.de>, marius <at> gnu.org
Subject: [PATCH v2 1/1] build-system/pyproject: Do not wrap native-inputs.
Date: Wed, 18 Jan 2023 09:00:35 -0500
Fixes <https://issues.guix.gnu.org/25235>.

* guix/build/pyproject-build-system.scm (wrap) [native-inputs]: New argument.
Filter out native inputs from the values in GUIX_PYTHONPATH.

---

Changes in v2:
- Add missing copyright line
- Rework wrap phase to avoid removing inputs found in both native-inputs and inputs
- Enclose wrap computations in an 'unless' form and streamline

 guix/build/pyproject-build-system.scm | 39 ++++++++++++++++-----------
 1 file changed, 24 insertions(+), 15 deletions(-)

diff --git a/guix/build/pyproject-build-system.scm b/guix/build/pyproject-build-system.scm
index a66c1fb34a..9da86bfc54 100644
--- a/guix/build/pyproject-build-system.scm
+++ b/guix/build/pyproject-build-system.scm
@@ -1,6 +1,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2021 Lars-Dominik Braun <lars <at> 6xq.net>
 ;;; Copyright © 2022 Marius Bakke <marius <at> gnu.org>
+;;; Copyright © 2023 Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -445,7 +446,7 @@ (define* (add-install-to-pythonpath #:key native-inputs outputs
   "A phase that just wraps the 'add-installed-pythonpath' procedure."
   (add-installed-pythonpath native-inputs outputs))
 
-(define* (wrap #:key inputs outputs #:allow-other-keys)
+(define* (wrap #:key native-inputs inputs outputs #:allow-other-keys)
   (define (list-of-files dir)
     (find-files dir (lambda (file stat)
                       (and (eq? 'regular (stat:type stat))
@@ -458,20 +459,28 @@ (define bindirs
                          (string-append dir "/sbin"))))
                 outputs))
 
-  ;; Do not require "guile" to be present in the package inputs
-  ;; even when there is nothing to wrap.
-  ;; Also, calculate (guile) only once to prevent some I/O.
-  (define %guile (delay (search-input-file inputs "bin/guile")))
-  (define (guile) (force %guile))
-
-  (let* ((var `("GUIX_PYTHONPATH" prefix
-                ,(search-path-as-string->list
-                  (or (getenv "GUIX_PYTHONPATH") "")))))
-    (for-each (lambda (dir)
-                (let ((files (list-of-files dir)))
-                  (for-each (cut wrap-script <> #:guile (guile) var)
-                            files)))
-              bindirs)))
+  (unless (null? bindirs)
+    (let* ((guile (search-input-file inputs "bin/guile"))
+           (native-input-dirs (match native-inputs
+                                (((_ . dir) ...)
+                                 dir)))
+           (input-dirs (match inputs
+                         (((_ . dir) ...)
+                          dir)))
+           (build-inputs (lset-difference string=? native-input-dirs
+                                          input-dirs))
+           ;; A build input is an input listed in native-inputs and NOT in
+           ;; inputs.
+           (build-input? (lambda (x)
+                           (any (cut string-prefix? <> x) build-inputs)))
+           (var `("GUIX_PYTHONPATH" prefix
+                  ,(remove build-input? (search-path-as-string->list
+                                         (or (getenv "GUIX_PYTHONPATH") ""))))))
+      (for-each (lambda (dir)
+                  (let ((files (list-of-files dir)))
+                    (for-each (cut wrap-script <> #:guile guile var)
+                              files)))
+                bindirs))))
 
 (define* (rename-pth-file #:key name native-inputs outputs #:allow-other-keys
                           #:rest args)
-- 
2.39.1





Information forwarded to bug-guix <at> gnu.org:
bug#25235; Package guix. (Mon, 23 Jan 2023 10:28:02 GMT) Full text and rfc822 format available.

Message #64 received at 25235 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Cc: arunisaac <at> systemreboot.net, 25235 <at> debbugs.gnu.org,
 Lars-Dominik Braun <lars <at> 6xq.net>, efraim <at> flashner.co.il, rekado <at> elephly.net,
 jgart <jgart <at> dismail.de>, marius <at> gnu.org
Subject: Re: [PATCH v2 1/1] build-system/pyproject: Do not wrap native-inputs.
Date: Mon, 23 Jan 2023 11:27:37 +0100
Hi,

Maxim Cournoyer <maxim.cournoyer <at> gmail.com> skribis:

> Fixes <https://issues.guix.gnu.org/25235>.
>
> * guix/build/pyproject-build-system.scm (wrap) [native-inputs]: New argument.
> Filter out native inputs from the values in GUIX_PYTHONPATH.

LGTM!

> -  ;; Do not require "guile" to be present in the package inputs
> -  ;; even when there is nothing to wrap.
> -  ;; Also, calculate (guile) only once to prevent some I/O.
> -  (define %guile (delay (search-input-file inputs "bin/guile")))
> -  (define (guile) (force %guile))

I still don’t see what branch this snippet is on, what am I missing?
:-)

Thanks,
Ludo’.




Information forwarded to bug-guix <at> gnu.org:
bug#25235; Package guix. (Mon, 23 Jan 2023 13:35:02 GMT) Full text and rfc822 format available.

Message #67 received at 25235 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: 25235 <at> debbugs.gnu.org
Cc: arunisaac <at> systemreboot.net, Maxim Cournoyer <maxim.cournoyer <at> gmail.com>,
 ludo <at> gnu.org, Lars-Dominik Braun <lars <at> 6xq.net>, efraim <at> flashner.co.il,
 rekado <at> elephly.net, jgart <jgart <at> dismail.de>, marius <at> gnu.org
Subject: [PATCH v3 0/1] build-system/pyproject: Do not wrap native-inputs.
Date: Mon, 23 Jan 2023 08:34:32 -0500
This fixes a longstanding issue.  The fix is made trivial by the changes made
in #60847 adding cross-compilation support for the pyproject build system.
Additionally, it's based on #60868, which touches the wrap phase to use
wrap-script instead of wrap-program.

Changes in v3:
- Rebase

Changes in v2:
- Add missing copyright line
- Rework wrap phase to avoid removing inputs found in both native-inputs and inputs
- Enclose wrap computations in an 'unless' form and streamline

Maxim Cournoyer (1):
  build-system/pyproject: Do not wrap native-inputs.

 guix/build/pyproject-build-system.scm | 39 ++++++++++++++++-----------
 1 file changed, 24 insertions(+), 15 deletions(-)


base-commit: 490e386e10ca2b7496c3da7ed85299ed3e9c2404
-- 
2.39.1





Information forwarded to bug-guix <at> gnu.org:
bug#25235; Package guix. (Mon, 23 Jan 2023 13:35:02 GMT) Full text and rfc822 format available.

Message #70 received at 25235 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: 25235 <at> debbugs.gnu.org
Cc: arunisaac <at> systemreboot.net, Maxim Cournoyer <maxim.cournoyer <at> gmail.com>,
 ludo <at> gnu.org, Lars-Dominik Braun <lars <at> 6xq.net>, efraim <at> flashner.co.il,
 rekado <at> elephly.net, jgart <jgart <at> dismail.de>, marius <at> gnu.org
Subject: [PATCH v3 1/1] build-system/pyproject: Do not wrap native-inputs.
Date: Mon, 23 Jan 2023 08:34:33 -0500
Fixes <https://issues.guix.gnu.org/25235>.

* guix/build/pyproject-build-system.scm (wrap) [native-inputs]: New argument.
Filter out native inputs from the values in GUIX_PYTHONPATH.

---

Changes in v3:
- Rebase

Changes in v2:
- Add missing copyright line
- Rework wrap phase to avoid removing inputs found in both native-inputs and inputs
- Enclose wrap computations in an 'unless' form and streamline

 guix/build/pyproject-build-system.scm | 39 ++++++++++++++++-----------
 1 file changed, 24 insertions(+), 15 deletions(-)

diff --git a/guix/build/pyproject-build-system.scm b/guix/build/pyproject-build-system.scm
index a66c1fb34a..9da86bfc54 100644
--- a/guix/build/pyproject-build-system.scm
+++ b/guix/build/pyproject-build-system.scm
@@ -1,6 +1,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2021 Lars-Dominik Braun <lars <at> 6xq.net>
 ;;; Copyright © 2022 Marius Bakke <marius <at> gnu.org>
+;;; Copyright © 2023 Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -445,7 +446,7 @@ (define* (add-install-to-pythonpath #:key native-inputs outputs
   "A phase that just wraps the 'add-installed-pythonpath' procedure."
   (add-installed-pythonpath native-inputs outputs))
 
-(define* (wrap #:key inputs outputs #:allow-other-keys)
+(define* (wrap #:key native-inputs inputs outputs #:allow-other-keys)
   (define (list-of-files dir)
     (find-files dir (lambda (file stat)
                       (and (eq? 'regular (stat:type stat))
@@ -458,20 +459,28 @@ (define bindirs
                          (string-append dir "/sbin"))))
                 outputs))
 
-  ;; Do not require "guile" to be present in the package inputs
-  ;; even when there is nothing to wrap.
-  ;; Also, calculate (guile) only once to prevent some I/O.
-  (define %guile (delay (search-input-file inputs "bin/guile")))
-  (define (guile) (force %guile))
-
-  (let* ((var `("GUIX_PYTHONPATH" prefix
-                ,(search-path-as-string->list
-                  (or (getenv "GUIX_PYTHONPATH") "")))))
-    (for-each (lambda (dir)
-                (let ((files (list-of-files dir)))
-                  (for-each (cut wrap-script <> #:guile (guile) var)
-                            files)))
-              bindirs)))
+  (unless (null? bindirs)
+    (let* ((guile (search-input-file inputs "bin/guile"))
+           (native-input-dirs (match native-inputs
+                                (((_ . dir) ...)
+                                 dir)))
+           (input-dirs (match inputs
+                         (((_ . dir) ...)
+                          dir)))
+           (build-inputs (lset-difference string=? native-input-dirs
+                                          input-dirs))
+           ;; A build input is an input listed in native-inputs and NOT in
+           ;; inputs.
+           (build-input? (lambda (x)
+                           (any (cut string-prefix? <> x) build-inputs)))
+           (var `("GUIX_PYTHONPATH" prefix
+                  ,(remove build-input? (search-path-as-string->list
+                                         (or (getenv "GUIX_PYTHONPATH") ""))))))
+      (for-each (lambda (dir)
+                  (let ((files (list-of-files dir)))
+                    (for-each (cut wrap-script <> #:guile guile var)
+                              files)))
+                bindirs))))
 
 (define* (rename-pth-file #:key name native-inputs outputs #:allow-other-keys
                           #:rest args)
-- 
2.39.1





Information forwarded to bug-guix <at> gnu.org:
bug#25235; Package guix. (Mon, 23 Jan 2023 13:39:02 GMT) Full text and rfc822 format available.

Message #73 received at 25235 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: arunisaac <at> systemreboot.net, 25235 <at> debbugs.gnu.org,
 Lars-Dominik Braun <lars <at> 6xq.net>, efraim <at> flashner.co.il, rekado <at> elephly.net,
 jgart <jgart <at> dismail.de>, marius <at> gnu.org
Subject: Re: [PATCH v2 1/1] build-system/pyproject: Do not wrap native-inputs.
Date: Mon, 23 Jan 2023 08:38:11 -0500
Hi Ludovic,

Ludovic Courtès <ludo <at> gnu.org> writes:

> Hi,
>
> Maxim Cournoyer <maxim.cournoyer <at> gmail.com> skribis:
>
>> Fixes <https://issues.guix.gnu.org/25235>.
>>
>> * guix/build/pyproject-build-system.scm (wrap) [native-inputs]: New argument.
>> Filter out native inputs from the values in GUIX_PYTHONPATH.
>
> LGTM!
>
>> -  ;; Do not require "guile" to be present in the package inputs
>> -  ;; even when there is nothing to wrap.
>> -  ;; Also, calculate (guile) only once to prevent some I/O.
>> -  (define %guile (delay (search-input-file inputs "bin/guile")))
>> -  (define (guile) (force %guile))
>
> I still don’t see what branch this snippet is on, what am I missing?
> :-)

It's based on https://issues.guix.gnu.org/60868, itself based on
https://issues.guix.gnu.org/60847.  I've tried to express this
relationship by marking the dependent changes blocked by the submissions
they are based on.

I've just rebased the 3 submissions and added you in CC; hopefully that
helps!

-- 
Thanks,
Maxim




This bug report was last modified 1 year and 66 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.