GNU bug report logs - #26987
guix pull without guile-ssh.

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 26987 in the body.
You can then email your comments to 26987 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <m.othacehe <at> gmail.com>
To: bug-guix <at> gnu.org
Subject: guix pull without guile-ssh.
Date: Thu, 18 May 2017 21:53:34 +0200

Hi,

If Guile-SSH is not detected, depends-on-guile-ssh? will be called here:

--8<---------------cut here---------------start------------->8---
(guix build pull)

(let* ((files (remove (if (false-if-exception
                           (resolve-interface '(ssh session)))
                          (const #f)
                        depends-on-guile-ssh?)
                      (all-scheme-files out)))
--8<---------------cut here---------------end--------------->8---

Thus, source-module-closure is called with a file. However,
source-module-closure is expecting a module list like '((guix ...)).

This result in the following error while running guix pull:

--8<---------------cut here---------------start------------->8---
In ./guix/modules.scm:
    118:4  0 (module-closure _ #:select? _ #:dependencies _)

./guix/modules.scm:118:4: In procedure module-closure:
./guix/modules.scm:118:4: Throw to key `match-error' with args `("match" "no matching pattern" "/gnu/store/0slxknjvjwhp31qryv42419vqwjrjifb-guix-latest/guix.scm")'.
builder for `/gnu/store/lq6ga13k75ljrdnqam9jm1q92cv4g0pm-guix-latest.drv' failed with exit code 1
--8<---------------cut here---------------end--------------->8---

I'm not sure how to fix it.

Thanks,

Mathieu

Message #8 received at 26987 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <m.othacehe <at> gmail.com>
To: 26987 <at> debbugs.gnu.org
Cc: Mathieu Othacehe <m.othacehe <at> gmail.com>
Subject: [PATCH 0/2] Fix compilation list construction.
Date: Fri, 19 May 2017 09:56:21 +0200

Hi,

Here are two patches to fix compilation list construction.
It seems to work but I'm higly unsure this is the right
thing to do.

Plus, while testing I could not add #:use-modules (guix discovery)
and had to write that instead :

((@@ (guix discovery) scheme-modules) directory)

Thanks,

Mathieu

Mathieu Othacehe (2):
  build: pull: Fix compilation list construction.
  guix: modules: Export module-name->file-name.

 guix/build/pull.scm | 31 ++++++++-----------------------
 guix/modules.scm    |  1 +
 2 files changed, 9 insertions(+), 23 deletions(-)

-- 
2.13.0

Message #11 received at 26987 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <m.othacehe <at> gmail.com>
To: 26987 <at> debbugs.gnu.org
Cc: Mathieu Othacehe <m.othacehe <at> gmail.com>
Subject: [PATCH 1/2] guix: modules: Export module-name->file-name.
Date: Fri, 19 May 2017 09:56:22 +0200

* guix/modules.scm (module-name->file-name): Export it.
---
 guix/modules.scm | 1 +
 1 file changed, 1 insertion(+)

diff --git a/guix/modules.scm b/guix/modules.scm
index 8c63f21a9..26c38e8d2 100644
--- a/guix/modules.scm
+++ b/guix/modules.scm
@@ -23,6 +23,7 @@
   #:use-module (ice-9 match)
   #:export (source-module-closure
             live-module-closure
+            module-name->file-name
             guix-module-name?))
 
 ;;; Commentary:
-- 
2.13.0

Message #14 received at 26987 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <m.othacehe <at> gmail.com>
To: 26987 <at> debbugs.gnu.org
Cc: Mathieu Othacehe <m.othacehe <at> gmail.com>
Subject: [PATCH 2/2] build: pull: Fix compilation list construction.
Date: Fri, 19 May 2017 09:56:23 +0200

* guix/build/pull.scm (depends-on-guile-ssh?): Remove.
(all-scheme-files): Use "scheme-modules" to detect all loadable
modules in given directory and rebuild a file list from those modules.
---
 guix/build/pull.scm | 31 ++++++++-----------------------
 1 file changed, 8 insertions(+), 23 deletions(-)

diff --git a/guix/build/pull.scm b/guix/build/pull.scm
index d2e0404b1..e6ec71a54 100644
--- a/guix/build/pull.scm
+++ b/guix/build/pull.scm
@@ -1,6 +1,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2013, 2014, 2016, 2017 Ludovic Courtès <ludo <at> gnu.org>
 ;;; Copyright © 2015 Taylan Ulrich Bayırlı/Kammer <taylanbayirli <at> gmail.com>
+;;; Copyright © 2017 Mathieu Othacehe <m.othacehe <at> gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -36,26 +37,14 @@
 ;;;
 ;;; Code:
 
-(define (depends-on-guile-ssh? file)
-  "Return true if FILE is a Scheme source file that depends, directly or
-indirectly, on Guile-SSH."
-  (find (match-lambda
-          (('ssh _ ...) #t)
-          (_ #f))
-        (source-module-closure file #:select? (const #t))))
-
 (define (all-scheme-files directory)
   "Return a sorted list of Scheme files found in DIRECTORY."
-  ;; Load guix/ modules before gnu/ modules to get somewhat steadier
-  ;; progress reporting.
-  (sort (filter (cut string-suffix? ".scm" <>)
-                (find-files directory "\\.scm"))
-        (let ((guix (string-append directory "/guix"))
-              (gnu  (string-append directory "/gnu")))
-          (lambda (a b)
-            (or (and (string-prefix? guix a)
-                     (string-prefix? gnu  b))
-                (string<? a b))))))
+  (let ((modules (map module-name
+		      ((@@ (guix discovery) scheme-modules) directory))))
+    (map (lambda (module)
+           (string-append directory "/"
+                          (module-name->file-name module)))
+        modules)))
 
 (cond-expand
   (guile-2.2 (use-modules (language tree-il optimize)
@@ -145,11 +134,7 @@ containing the source code.  Write any debugging output to DEBUG-PORT."
     ;; Compile the .scm files.  Load all the files before compiling them to
     ;; work around <http://bugs.gnu.org/15602> (FIXME).
     ;; Filter out files depending on Guile-SSH when Guile-SSH is missing.
-    (let* ((files (remove (if (false-if-exception
-                               (resolve-interface '(ssh session)))
-                              (const #f)
-                              depends-on-guile-ssh?)
-                          (all-scheme-files out)))
+    (let* ((files (all-scheme-files out))
            (total (length files)))
       (let loop ((files files)
                  (completed 0))
-- 
2.13.0

Message #17 received at 26987 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Mathieu Othacehe <m.othacehe <at> gmail.com>
Cc: 26987 <at> debbugs.gnu.org
Subject: Re: bug#26987: [PATCH 1/2] guix: modules: Export
 module-name->file-name.
Date: Fri, 19 May 2017 10:18:11 +0200

Mathieu Othacehe <m.othacehe <at> gmail.com> skribis:

> * guix/modules.scm (module-name->file-name): Export it.

OK!

Message #20 received at 26987 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Mathieu Othacehe <m.othacehe <at> gmail.com>
Cc: 26987 <at> debbugs.gnu.org
Subject: Re: bug#26987: [PATCH 2/2] build: pull: Fix compilation list
 construction.
Date: Fri, 19 May 2017 14:58:48 +0200

[Message part 1 (text/plain, inline)]

Mathieu Othacehe <m.othacehe <at> gmail.com> skribis:

> * guix/build/pull.scm (depends-on-guile-ssh?): Remove.
> (all-scheme-files): Use "scheme-modules" to detect all loadable
> modules in given directory and rebuild a file list from those modules.
> ---
>  guix/build/pull.scm | 31 ++++++++-----------------------
>  1 file changed, 8 insertions(+), 23 deletions(-)
>
> diff --git a/guix/build/pull.scm b/guix/build/pull.scm
> index d2e0404b1..e6ec71a54 100644
> --- a/guix/build/pull.scm
> +++ b/guix/build/pull.scm
> @@ -1,6 +1,7 @@
>  ;;; GNU Guix --- Functional package management for GNU
>  ;;; Copyright © 2013, 2014, 2016, 2017 Ludovic Courtès <ludo <at> gnu.org>
>  ;;; Copyright © 2015 Taylan Ulrich Bayırlı/Kammer <taylanbayirli <at> gmail.com>
> +;;; Copyright © 2017 Mathieu Othacehe <m.othacehe <at> gmail.com>
>  ;;;
>  ;;; This file is part of GNU Guix.
>  ;;;
> @@ -36,26 +37,14 @@
>  ;;;
>  ;;; Code:
>  
> -(define (depends-on-guile-ssh? file)
> -  "Return true if FILE is a Scheme source file that depends, directly or
> -indirectly, on Guile-SSH."
> -  (find (match-lambda
> -          (('ssh _ ...) #t)
> -          (_ #f))
> -        (source-module-closure file #:select? (const #t))))
> -
>  (define (all-scheme-files directory)
>    "Return a sorted list of Scheme files found in DIRECTORY."
> -  ;; Load guix/ modules before gnu/ modules to get somewhat steadier
> -  ;; progress reporting.
> -  (sort (filter (cut string-suffix? ".scm" <>)
> -                (find-files directory "\\.scm"))
> -        (let ((guix (string-append directory "/guix"))
> -              (gnu  (string-append directory "/gnu")))
> -          (lambda (a b)
> -            (or (and (string-prefix? guix a)
> -                     (string-prefix? gnu  b))
> -                (string<? a b))))))
> +  (let ((modules (map module-name
> +		      ((@@ (guix discovery) scheme-modules) directory))))
> +    (map (lambda (module)
> +           (string-append directory "/"
> +                          (module-name->file-name module)))
> +        modules)))
>  
>  (cond-expand
>    (guile-2.2 (use-modules (language tree-il optimize)
> @@ -145,11 +134,7 @@ containing the source code.  Write any debugging output to DEBUG-PORT."
>      ;; Compile the .scm files.  Load all the files before compiling them to
>      ;; work around <http://bugs.gnu.org/15602> (FIXME).
>      ;; Filter out files depending on Guile-SSH when Guile-SSH is missing.
> -    (let* ((files (remove (if (false-if-exception
> -                               (resolve-interface '(ssh session)))
> -                              (const #f)
> -                              depends-on-guile-ssh?)
> -                          (all-scheme-files out)))
> +    (let* ((files (all-scheme-files out))
>             (total (length files)))

I think this patch does to thing: (1) fixing the bug at hand, and (2)
using (guix discovery).

For (1), I think this should work:

[Message part 2 (text/x-patch, inline)]

diff --git a/guix/build/pull.scm b/guix/build/pull.scm
index d2e0404b1..84629f6b0 100644
--- a/guix/build/pull.scm
+++ b/guix/build/pull.scm
@@ -39,10 +39,15 @@
 (define (depends-on-guile-ssh? file)
   "Return true if FILE is a Scheme source file that depends, directly or
 indirectly, on Guile-SSH."
+  (let ((module (call-with-input-file file
+                  (lambda (port)
+                    (match (read port)
+                      (('define-module name _ ...)
+                       name))))))
     (find (match-lambda
             (('ssh _ ...) #t)
             (_ #f))
-        (source-module-closure file #:select? (const #t))))
+          (source-module-closure module #:select? (const #t)))))
 
 (define (all-scheme-files directory)
   "Return a sorted list of Scheme files found in DIRECTORY."

[Message part 3 (text/plain, inline)]

WDYT?

Ludo’.

Message #23 received at 26987 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <m.othacehe <at> gmail.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 26987 <at> debbugs.gnu.org
Subject: Re: bug#26987: [PATCH 2/2] build: pull: Fix compilation list
 construction.
Date: Fri, 19 May 2017 15:55:31 +0200

Hi Ludo,

It seems better but I can't get it to work neither :

>      (find (match-lambda
>              (('ssh _ ...) #t)
>              (_ #f))
> -        (source-module-closure file #:select? (const #t))))
> +          (source-module-closure module #:select? (const #t)))))

                                       ^
                                       we need a list here.
>  
>  (define (all-scheme-files directory)
>    "Return a sorted list of Scheme files found in DIRECTORY."
>
> WDYT?

So with

--8<---------------cut here---------------start------------->8---
(define (depends-on-guile-ssh? file)
  "Return true if FILE is a Scheme source file that depends, directly or
indirectly, on Guile-SSH."
  (let ((module (call-with-input-file file
                  (lambda (port)
                    (match (read port)
                      (('define-module name _ ...)
                       name))))))
    (find (match-lambda
            (('ssh _ ...) #t)
            (_ #f))
          (source-module-closure (list module) #:select? (const #t)))))
--8<---------------cut here---------------end--------------->8---

I get,

--8<---------------cut here---------------start------------->8---
((@@ (guix build pull) depends-on-guile-ssh?) "/home/mathieu/guix/guix/ssh.scm")

ERROR: In procedure open-file:
ERROR: Wrong type (expecting string): #f
--8<---------------cut here---------------end--------------->8---

I'm having a hard time tring to understand ,trace output.

Mathieu

Message #26 received at 26987 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <m.othacehe <at> gmail.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 26987 <at> debbugs.gnu.org
Subject: Re: bug#26987: [PATCH 2/2] build: pull: Fix compilation list
 construction.
Date: Fri, 19 May 2017 16:26:17 +0200

> I'm having a hard time tring to understand ,trace output.

I get why, it's because guile-ssh is not in my %load-path. So loading
"ssh/session.scm" fails in "source-module-dependencies".

It's a bit of a vicious circle here : if resolve-interface '(ssh
session) fails, depends-on-guile-ssh? will fail too.

I'm note sure we can depend on source-module-closure to do the job here.

Mathieu

Message #29 received at 26987 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Mathieu Othacehe <m.othacehe <at> gmail.com>
Cc: 26987 <at> debbugs.gnu.org
Subject: Re: bug#26987: [PATCH 2/2] build: pull: Fix compilation list
 construction.
Date: Fri, 19 May 2017 17:41:42 +0200

Mathieu Othacehe <m.othacehe <at> gmail.com> skribis:

>> I'm having a hard time tring to understand ,trace output.
>
> I get why, it's because guile-ssh is not in my %load-path. So loading
> "ssh/session.scm" fails in "source-module-dependencies".
>
> It's a bit of a vicious circle here : if resolve-interface '(ssh
> session) fails, depends-on-guile-ssh? will fail too.
>
> I'm note sure we can depend on source-module-closure to do the job here.

Oh, got it.  Hmm, needs more thought.

Ludo’.

Message #36 received at 26987-done <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Mathieu Othacehe <m.othacehe <at> gmail.com>
Cc: 26987-done <at> debbugs.gnu.org
Subject: Re: bug#26987: [PATCH 2/2] build: pull: Fix compilation list
 construction.
Date: Thu, 25 May 2017 14:33:04 +0200

Mathieu Othacehe <m.othacehe <at> gmail.com> skribis:

>> I'm having a hard time tring to understand ,trace output.
>
> I get why, it's because guile-ssh is not in my %load-path. So loading
> "ssh/session.scm" fails in "source-module-dependencies".
>
> It's a bit of a vicious circle here : if resolve-interface '(ssh
> session) fails, depends-on-guile-ssh? will fail too.
>
> I'm note sure we can depend on source-module-closure to do the job here.

I believe this is fixed by commit
20ed093977cc80ba1729c38e05ae7955a38069a6, which follows a modification
to the ‘source-module-closure’ so that callers can catch
missing-dependency errors.

Please let me know what you think!

After that, you should be able to reinstate the (guix git) commit,
provided build-aux/build-self.scm does:

  (define guile-git
    (package-for-current-guile "guile-git"
                               "guile2.2-git"
                               "guile2.0-git"))


and passes that down to ‘build-guix’.

Thanks,
Ludo’.

Message #39 received at 26987-done <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <m.othacehe <at> gmail.com>
To: Ludovic Courtès <ludo <at> gnu.org>, Amirouche
 <amirouche <at> hypermove.net>
Cc: 26987-done <at> debbugs.gnu.org
Subject: Re: bug#26987: [PATCH 2/2] build: pull: Fix compilation list
 construction.
Date: Fri, 02 Jun 2017 11:16:04 +0200

Hi Ludo,

> I believe this is fixed by commit
> 20ed093977cc80ba1729c38e05ae7955a38069a6, which follows a modification
> to the ‘source-module-closure’ so that callers can catch
> missing-dependency errors.
>
> Please let me know what you think!

It seems ok, thanks for fixing it !

>
> After that, you should be able to reinstate the (guix git) commit,
> provided build-aux/build-self.scm does:
>
>   (define guile-git
>     (package-for-current-guile "guile-git"
>                                "guile2.2-git"
>                                "guile2.0-git"))
                                        ^
                           for guile2.0-git,
we need to define guile2.0-bytestructures, so I tried to package it,
and I get the following error :

--8<---------------cut here---------------start------------->8---
ice-9/boot-9.scm:109:20: In procedure #<procedure 9dbe40 at ice-9/boot-9.scm:100:6 (thrown-k . 
ice-9/boot-9.scm:109:20: re-exporting local variable: cstring-pointer
--8<---------------cut here---------------end--------------->8---

It is something introduced in bytestructures commit 91d042e, that works
on 2.2 but not on 2.0, I have no idea why !

Amirouche any idea ?

Thanks,

Mathieu

Message #42 received at 26987-done <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Mathieu Othacehe <m.othacehe <at> gmail.com>
Cc: 26987-done <at> debbugs.gnu.org, Amirouche <amirouche <at> hypermove.net>
Subject: Re: bug#26987: [PATCH 2/2] build: pull: Fix compilation list
 construction.
Date: Fri, 02 Jun 2017 17:09:47 +0200

Hi Mathieu,

Mathieu Othacehe <m.othacehe <at> gmail.com> skribis:

>> I believe this is fixed by commit
>> 20ed093977cc80ba1729c38e05ae7955a38069a6, which follows a modification
>> to the ‘source-module-closure’ so that callers can catch
>> missing-dependency errors.
>>
>> Please let me know what you think!
>
> It seems ok, thanks for fixing it !
>
>>
>> After that, you should be able to reinstate the (guix git) commit,
>> provided build-aux/build-self.scm does:
>>
>>   (define guile-git
>>     (package-for-current-guile "guile-git"
>>                                "guile2.2-git"
>>                                "guile2.0-git"))
>                                         ^
>                            for guile2.0-git,
> we need to define guile2.0-bytestructures, so I tried to package it,
> and I get the following error :
>
> ice-9/boot-9.scm:109:20: In procedure #<procedure 9dbe40 at ice-9/boot-9.scm:100:6 (thrown-k . 
> ice-9/boot-9.scm:109:20: re-exporting local variable: cstring-pointer
>
> It is something introduced in bytestructures commit 91d042e, that works
> on 2.2 but not on 2.0, I have no idea why !

It would be nice to have guile2.0-git, but note that it does not *have*
to exist: ‘package-for-current-guile’ returns #f if none of the packages
exists, and (guix build pull) skips modules that lack their
dependencies.

So I would first suggest to re-add (guix git), and later see with
Amirouche et al. what could be done to get Guile-Git on 2.0.

Does that make sense?

Thanks,
Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.