GNU bug report logs - #26023
[PATCH] gexp: Expose functions to allow creating derivation builders.

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 26023 in the body.
You can then email your comments to 26023 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Roel Janssen <roel <at> gnu.org>
To: guix-patches <at> gnu.org
Subject: [PATCH] gexp: Expose functions to allow creating derivation builders.
Date: Wed, 08 Mar 2017 11:21:47 +0100

Dear Guix,

I would like to add the following two functions to the public interface
of the gexp module.

This makes writing custom derivation builders easier for external
projects.

Kind regards,
Roel Janssen


>From b70e95b40876fd372fc44958e6102ad39ffc2478 Mon Sep 17 00:00:00 2001
From: Roel Janssen <roel <at> gnu.org>
Date: Wed, 8 Mar 2017 11:17:13 +0100
Subject: [PATCH] gexp: Expose functions to allow creating derivation builders.

* guix/gexp.scm:  Add load-path-expression and gexp-modules to the public
  interface of the module.
---
 guix/gexp.scm | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/guix/gexp.scm b/guix/gexp.scm
index 1f7fbef0a..d11ed177f 100644
--- a/guix/gexp.scm
+++ b/guix/gexp.scm
@@ -68,6 +68,9 @@
             file-append-base
             file-append-suffix
 
+            load-path-expression
+            gexp-modules
+
             gexp->derivation
             gexp->file
             gexp->script
-- 
2.12.0

Message #8 received at 26023 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Roel Janssen <roel <at> gnu.org>
Cc: 26023 <at> debbugs.gnu.org
Subject: Re: bug#26023: [PATCH] gexp: Expose functions to allow creating
 derivation builders.
Date: Wed, 08 Mar 2017 15:18:02 +0100

Hello!

Roel Janssen <roel <at> gnu.org> skribis:

> I would like to add the following two functions to the public interface
> of the gexp module.
>
> This makes writing custom derivation builders easier for external
> projects.

Out of curiosity, how do you use them?

I’m asking because they’re not too bad but not too nice an interface
either, especially ‘load-path-expression’, and I’d expect the use cases
to be covered by ‘gexp->script’ & ‘gexp->file’.

Ludo’.

Message #11 received at 26023 <at> debbugs.gnu.org (full text, mbox):

From: Roel Janssen <roel <at> gnu.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 26023 <at> debbugs.gnu.org
Subject: Re: bug#26023: [PATCH] gexp: Expose functions to allow creating
 derivation builders.
Date: Wed, 08 Mar 2017 15:26:27 +0100

Ludovic Courtès writes:

> Hello!
>
> Roel Janssen <roel <at> gnu.org> skribis:
>
>> I would like to add the following two functions to the public interface
>> of the gexp module.
>>
>> This makes writing custom derivation builders easier for external
>> projects.
>
> Out of curiosity, how do you use them?
>
> I’m asking because they’re not too bad but not too nice an interface
> either, especially ‘load-path-expression’, and I’d expect the use cases
> to be covered by ‘gexp->script’ & ‘gexp->file’.
>
> Ludo’.

Here's how I use it:

(define* (process->bash-engine-derivation proc #:key (guile (default-guile)))
  "Return an executable script that runs the PROCEDURE described in PROC, with
PROCEDURE's imported modules in its search path."
  (let ((name (process-full-name proc))
        (exp (process-procedure proc))
        (out (process-output-path proc)))
    (let ((out-str (if out (format #f "(setenv \"out\" ~s)" out) "")))
      (mlet %store-monad ((set-load-path
                           (load-path-expression (gexp-modules exp))))
        (gexp->derivation
         name
         (gexp
          (call-with-output-file (ungexp output)
            (lambda (port)
              (use-modules (ice-9 pretty-print))
              (format port "#!~a/bin/bash~%" (ungexp bash))
              ;; Now that we've written all of the shell code,
              ;; We can start writing the Scheme code.
              ;; We rely on Bash for this to work.
              (format port "read -d '' CODE <<EOF~%")
              ;; The destination can be outside of the store.
              ;; TODO: We have to mount this location when building inside
              ;; a container.
              (format port "~a" (ungexp out-str))
              (format port
                      "~%;; Code to create a proper Guile environment.~%~a~%"
                      (with-output-to-string
                        (lambda _ (pretty-print '(ungexp set-load-path)))))
              (format port ";; Actual code from the procedure.~%~a~%"
                      (with-output-to-string
                        (lambda _ (pretty-print '(ungexp exp)))))
              (format port "EOF~%")
              (format port "~a/bin/guile -c \"$CODE\"~%" (ungexp guile))
              (chmod port #o555)))))))))

If you have a suggestion on how to avoid using them, I'd love to hear it.

Kind regards,
Roel Janssen

Message #14 received at 26023 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Roel Janssen <roel <at> gnu.org>
Cc: 26023 <at> debbugs.gnu.org
Subject: Re: bug#26023: [PATCH] gexp: Expose functions to allow creating
 derivation builders.
Date: Wed, 08 Mar 2017 22:30:46 +0100

Roel Janssen <roel <at> gnu.org> skribis:

> Ludovic Courtès writes:
>
>> Hello!
>>
>> Roel Janssen <roel <at> gnu.org> skribis:
>>
>>> I would like to add the following two functions to the public interface
>>> of the gexp module.
>>>
>>> This makes writing custom derivation builders easier for external
>>> projects.
>>
>> Out of curiosity, how do you use them?
>>
>> I’m asking because they’re not too bad but not too nice an interface
>> either, especially ‘load-path-expression’, and I’d expect the use cases
>> to be covered by ‘gexp->script’ & ‘gexp->file’.
>>
>> Ludo’.
>
> Here's how I use it:
>
> (define* (process->bash-engine-derivation proc #:key (guile (default-guile)))
>   "Return an executable script that runs the PROCEDURE described in PROC, with
> PROCEDURE's imported modules in its search path."
>   (let ((name (process-full-name proc))
>         (exp (process-procedure proc))
>         (out (process-output-path proc)))
>     (let ((out-str (if out (format #f "(setenv \"out\" ~s)" out) "")))
>       (mlet %store-monad ((set-load-path
>                            (load-path-expression (gexp-modules exp))))
>         (gexp->derivation
>          name
>          (gexp
>           (call-with-output-file (ungexp output)
>             (lambda (port)
>               (use-modules (ice-9 pretty-print))
>               (format port "#!~a/bin/bash~%" (ungexp bash))
>               ;; Now that we've written all of the shell code,
>               ;; We can start writing the Scheme code.
>               ;; We rely on Bash for this to work.
>               (format port "read -d '' CODE <<EOF~%")
>               ;; The destination can be outside of the store.
>               ;; TODO: We have to mount this location when building inside
>               ;; a container.
>               (format port "~a" (ungexp out-str))
>               (format port
>                       "~%;; Code to create a proper Guile environment.~%~a~%"
>                       (with-output-to-string
>                         (lambda _ (pretty-print '(ungexp set-load-path)))))
>               (format port ";; Actual code from the procedure.~%~a~%"
>                       (with-output-to-string
>                         (lambda _ (pretty-print '(ungexp exp)))))
>               (format port "EOF~%")
>               (format port "~a/bin/guile -c \"$CODE\"~%" (ungexp guile))
>               (chmod port #o555)))))))))
>
> If you have a suggestion on how to avoid using them, I'd love to hear it.

In theory you could put the Scheme code in a separate file (created with
‘gexp->script’), or avoid Bash altogether, but perhaps none of these is
an option?

Cheers,
Ludo’.

Message #17 received at 26023 <at> debbugs.gnu.org (full text, mbox):

From: Roel Janssen <roel <at> gnu.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 26023 <at> debbugs.gnu.org
Subject: Re: bug#26023: [PATCH] gexp: Expose functions to allow creating
 derivation builders.
Date: Wed, 08 Mar 2017 23:00:08 +0100

Ludovic Courtès writes:

> Roel Janssen <roel <at> gnu.org> skribis:
>
>> Ludovic Courtès writes:
>>
>>> Hello!
>>>
>>> Roel Janssen <roel <at> gnu.org> skribis:
>>>
>>>> I would like to add the following two functions to the public interface
>>>> of the gexp module.
>>>>
>>>> This makes writing custom derivation builders easier for external
>>>> projects.
>>>
>>> Out of curiosity, how do you use them?
>>>
>>> I’m asking because they’re not too bad but not too nice an interface
>>> either, especially ‘load-path-expression’, and I’d expect the use cases
>>> to be covered by ‘gexp->script’ & ‘gexp->file’.
>>>
>>> Ludo’.
>>
>> Here's how I use it:
>>
>> (define* (process->bash-engine-derivation proc #:key (guile (default-guile)))
>>   "Return an executable script that runs the PROCEDURE described in PROC, with
>> PROCEDURE's imported modules in its search path."
>>   (let ((name (process-full-name proc))
>>         (exp (process-procedure proc))
>>         (out (process-output-path proc)))
>>     (let ((out-str (if out (format #f "(setenv \"out\" ~s)" out) "")))
>>       (mlet %store-monad ((set-load-path
>>                            (load-path-expression (gexp-modules exp))))
>>         (gexp->derivation
>>          name
>>          (gexp
>>           (call-with-output-file (ungexp output)
>>             (lambda (port)
>>               (use-modules (ice-9 pretty-print))
>>               (format port "#!~a/bin/bash~%" (ungexp bash))
>>               ;; Now that we've written all of the shell code,
>>               ;; We can start writing the Scheme code.
>>               ;; We rely on Bash for this to work.
>>               (format port "read -d '' CODE <<EOF~%")
>>               ;; The destination can be outside of the store.
>>               ;; TODO: We have to mount this location when building inside
>>               ;; a container.
>>               (format port "~a" (ungexp out-str))
>>               (format port
>>                       "~%;; Code to create a proper Guile environment.~%~a~%"
>>                       (with-output-to-string
>>                         (lambda _ (pretty-print '(ungexp set-load-path)))))
>>               (format port ";; Actual code from the procedure.~%~a~%"
>>                       (with-output-to-string
>>                         (lambda _ (pretty-print '(ungexp exp)))))
>>               (format port "EOF~%")
>>               (format port "~a/bin/guile -c \"$CODE\"~%" (ungexp guile))
>>               (chmod port #o555)))))))))
>>
>> If you have a suggestion on how to avoid using them, I'd love to hear it.
>
> In theory you could put the Scheme code in a separate file (created with
> ‘gexp->script’), or avoid Bash altogether, but perhaps none of these is
> an option?

No, it's meant to build runnable scripts that can be run by a job
scheduling system (Sun Grid Engine).  It's configured to just run a bash
script (yes, really bash, not just a shell script) with some additional
syntax.  (For the sake of the example, I gave a shorter version that
"only" generates the regular bash script.

Perhaps we can come up with a better interface for the gexp module to
handle these cases.  I will give this some thought.

Kind regards,
Roel Janssen

Message #20 received at 26023 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Roel Janssen <roel <at> gnu.org>
Cc: 26023 <at> debbugs.gnu.org
Subject: Re: bug#26023: [PATCH] gexp: Expose functions to allow creating
 derivation builders.
Date: Thu, 09 Mar 2017 11:44:54 +0100

Hello!

Roel Janssen <roel <at> gnu.org> skribis:

> Ludovic Courtès writes:
>
>> Roel Janssen <roel <at> gnu.org> skribis:
>>
>>> Ludovic Courtès writes:
>>>
>>>> Hello!
>>>>
>>>> Roel Janssen <roel <at> gnu.org> skribis:
>>>>
>>>>> I would like to add the following two functions to the public interface
>>>>> of the gexp module.
>>>>>
>>>>> This makes writing custom derivation builders easier for external
>>>>> projects.
>>>>
>>>> Out of curiosity, how do you use them?
>>>>
>>>> I’m asking because they’re not too bad but not too nice an interface
>>>> either, especially ‘load-path-expression’, and I’d expect the use cases
>>>> to be covered by ‘gexp->script’ & ‘gexp->file’.
>>>>
>>>> Ludo’.
>>>
>>> Here's how I use it:
>>>
>>> (define* (process->bash-engine-derivation proc #:key (guile (default-guile)))
>>>   "Return an executable script that runs the PROCEDURE described in PROC, with
>>> PROCEDURE's imported modules in its search path."
>>>   (let ((name (process-full-name proc))
>>>         (exp (process-procedure proc))
>>>         (out (process-output-path proc)))
>>>     (let ((out-str (if out (format #f "(setenv \"out\" ~s)" out) "")))
>>>       (mlet %store-monad ((set-load-path
>>>                            (load-path-expression (gexp-modules exp))))
>>>         (gexp->derivation
>>>          name
>>>          (gexp
>>>           (call-with-output-file (ungexp output)
>>>             (lambda (port)
>>>               (use-modules (ice-9 pretty-print))
>>>               (format port "#!~a/bin/bash~%" (ungexp bash))
>>>               ;; Now that we've written all of the shell code,
>>>               ;; We can start writing the Scheme code.
>>>               ;; We rely on Bash for this to work.
>>>               (format port "read -d '' CODE <<EOF~%")
>>>               ;; The destination can be outside of the store.
>>>               ;; TODO: We have to mount this location when building inside
>>>               ;; a container.
>>>               (format port "~a" (ungexp out-str))
>>>               (format port
>>>                       "~%;; Code to create a proper Guile environment.~%~a~%"
>>>                       (with-output-to-string
>>>                         (lambda _ (pretty-print '(ungexp set-load-path)))))
>>>               (format port ";; Actual code from the procedure.~%~a~%"
>>>                       (with-output-to-string
>>>                         (lambda _ (pretty-print '(ungexp exp)))))
>>>               (format port "EOF~%")
>>>               (format port "~a/bin/guile -c \"$CODE\"~%" (ungexp guile))
>>>               (chmod port #o555)))))))))
>>>
>>> If you have a suggestion on how to avoid using them, I'd love to hear it.
>>
>> In theory you could put the Scheme code in a separate file (created with
>> ‘gexp->script’), or avoid Bash altogether, but perhaps none of these is
>> an option?
>
> No, it's meant to build runnable scripts that can be run by a job
> scheduling system (Sun Grid Engine).  It's configured to just run a bash
> script (yes, really bash, not just a shell script) with some additional
> syntax.  (For the sake of the example, I gave a shorter version that
> "only" generates the regular bash script.

Oh, I see.

> Perhaps we can come up with a better interface for the gexp module to
> handle these cases.  I will give this some thought.

In the meantime, you can go ahead and expose these two procedures, it’s
not that bad either.  Please add a reference to this issue in the commit
log.

Thanks,
Ludo’.

Message #25 received at 26023-done <at> debbugs.gnu.org (full text, mbox):

From: Roel Janssen <roel <at> gnu.org>
To: 26023-done <at> debbugs.gnu.org
Date: Thu, 09 Mar 2017 12:44:57 +0100

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.