GNU bug report logs - #54003
'local-file' fails to resolve relative file names with Guile 3.0.8

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 54003 in the body.
You can then email your comments to 54003 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Aleksandr Vityazev <avityazev <at> posteo.org>
To: bug-guix <at> gnu.org
Subject: local-file after Guile update
Date: Mon, 14 Feb 2022 21:50:08 +0000

Hi,

I have the following dotfiles directory structure where I store my Guix home
config:

|-etc
|
|-subdir
|   |
|   |-home


I often use the 'local-file' procedure in modules in the home directory. For
example in /subdir/home/shells.scm:

#+begin_src scheme :scheme guile :season guile
(simple-service 'add-zsh-files
                home-files-service-type
                `(("config/zsh/zshrc"
                   ,(local-file "../../etc/zsh/zshrc"))))
#+end_src
And it works, as expected.

The following example also worked before the
076e825dc5d585943ce820a279fffe4af09757fb (gnu: guile: Add version 3.0.8 as
'guile-3.0-latest') commit.

in /subdir/home/wm.scm: 
#+begin_src scheme :scheme guile :season guile
(service
 home-rofi-service-type
 (home-rofi-configuration
  (config-rasi
   `(,#~(format
         #f "@theme \"~a\""
         #$(local-file "../../etc/rofi/nord.rasi"))))))
#+end_src

After 076e825dc5d585943ce820a279fffe4af09757fb
using (local-file "../../etc/rofi/nord.rasi") and not
(local-file "etc/rofi/nord.rasi") causes an error:

guix home: error: canonicalize-path: No such file or
directory: "../../etc/rofi/nord.rasi".

The example that causes the error uses the original Guix home implementation,
which uses gexps.  When I figure out how to make a minimal reproducer that
does not depend on it, I will update this report.

-- 
Best regards,
Aleksandr Vityazev

Message #8 received at 54003 <at> debbugs.gnu.org (full text, mbox):

From: Liliana Marie Prikler <liliana.prikler <at> ist.tugraz.at>
To: Aleksandr Vityazev <avityazev <at> posteo.org>, 54003 <at> debbugs.gnu.org
Subject: Re: local-file after Guile update
Date: Tue, 15 Feb 2022 09:21:54 +0100

Hi Aleksandr

Am Montag, dem 14.02.2022 um 21:50 +0000 schrieb Aleksandr Vityazev:
> Hi,
> 
> I have the following dotfiles directory structure where I store my
> Guix home config:
> 
> > -etc
> > 
> > -subdir
> >   |
> >   |-home
> 
> 
> I often use the 'local-file' procedure in modules in the home
> directory. For example in /subdir/home/shells.scm:
> 
> #+begin_src scheme :scheme guile :season guile
> (simple-service 'add-zsh-files
>                 home-files-service-type
>                 `(("config/zsh/zshrc"
>                    ,(local-file "../../etc/zsh/zshrc"))))
> #+end_src
> And it works, as expected.
Be careful when invoking Hyrum's Law.  The meaning of this local-file
expression changes with the current working directory, which might be
different across invocations.  To avoid such issues, you can (almost?)
always use (current-file-name), which you might want to canonicalize
for the operations you're about to do.
Taking (dirname) thrice then brings you to $GUIX_HOME_CONFIG_ROOT,
which contains both etc and subdir.

> The following example also worked before the
> 076e825dc5d585943ce820a279fffe4af09757fb (gnu: guile: Add version
> 3.0.8 as
> 'guile-3.0-latest') commit.
> 
> in /subdir/home/wm.scm: 
> #+begin_src scheme :scheme guile :season guile
> (service
>  home-rofi-service-type
>  (home-rofi-configuration
>   (config-rasi
>    `(,#~(format
>          #f "@theme \"~a\""
>          #$(local-file "../../etc/rofi/nord.rasi"))))))
> #+end_src
> 
> After 076e825dc5d585943ce820a279fffe4af09757fb
> using (local-file "../../etc/rofi/nord.rasi") and not
> (local-file "etc/rofi/nord.rasi") causes an error:
> 
> guix home: error: canonicalize-path: No such file or
> directory: "../../etc/rofi/nord.rasi".
> 
> The example that causes the error uses the original Guix home
> implementation, which uses gexps.  When I figure out how to make a
> minimal reproducer that does not depend on it, I will update this
> report.
I don't think this is an issue particularly in Guix Home, but for a MWE
you could try only writing a bashrc, which ought to work with upstream
as well.  Use `guix home build` rather than `guix home reconfigure` so
as to not activate it.

Cheers

Message #11 received at 54003 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Aleksandr Vityazev <avityazev <at> posteo.org>
Cc: 54003 <at> debbugs.gnu.org
Subject: Re: bug#54003: local-file after Guile update
Date: Tue, 15 Feb 2022 16:00:11 +0100

Hi,

Aleksandr Vityazev <avityazev <at> posteo.org> skribis:

> The following example also worked before the
> 076e825dc5d585943ce820a279fffe4af09757fb (gnu: guile: Add version 3.0.8 as
> 'guile-3.0-latest') commit.
>
> in /subdir/home/wm.scm: 
>
> #+begin_src scheme :scheme guile :season guile
> (service
>  home-rofi-service-type
>  (home-rofi-configuration
>   (config-rasi
>    `(,#~(format
>          #f "@theme \"~a\""
>          #$(local-file "../../etc/rofi/nord.rasi"))))))
> #+end_src
>
> After 076e825dc5d585943ce820a279fffe4af09757fb
> using (local-file "../../etc/rofi/nord.rasi") and not
> (local-file "etc/rofi/nord.rasi") causes an error:
>
> guix home: error: canonicalize-path: No such file or
> directory: "../../etc/rofi/nord.rasi".

I’ve witnessed a similar problem that I worked around in
c334b7c52fe77b68a90b23fbac5c9de7337e607b.

Needs some investigation…

Ludo’.

Message #18 received at 54003 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 54003 <at> debbugs.gnu.org, Aleksandr Vityazev <avityazev <at> posteo.org>
Subject: Re: bug#54003: local-file after Guile update
Date: Tue, 15 Feb 2022 10:57:19 -0500

Hello,

Ludovic Courtès <ludo <at> gnu.org> writes:

> Hi,
>
> Aleksandr Vityazev <avityazev <at> posteo.org> skribis:
>
>> The following example also worked before the
>> 076e825dc5d585943ce820a279fffe4af09757fb (gnu: guile: Add version 3.0.8 as
>> 'guile-3.0-latest') commit.
>>
>> in /subdir/home/wm.scm: 
>>
>> #+begin_src scheme :scheme guile :season guile
>> (service
>>  home-rofi-service-type
>>  (home-rofi-configuration
>>   (config-rasi
>>    `(,#~(format
>>          #f "@theme \"~a\""
>>          #$(local-file "../../etc/rofi/nord.rasi"))))))
>> #+end_src
>>
>> After 076e825dc5d585943ce820a279fffe4af09757fb
>> using (local-file "../../etc/rofi/nord.rasi") and not
>> (local-file "etc/rofi/nord.rasi") causes an error:
>>
>> guix home: error: canonicalize-path: No such file or
>> directory: "../../etc/rofi/nord.rasi".
>
> I’ve witnessed a similar problem that I worked around in
> c334b7c52fe77b68a90b23fbac5c9de7337e607b.

Another example is the berlin configuration; attempting to reconfigure
it currently yields:

--8<---------------cut here---------------start------------->8---
$ sudo guix system reconfigure -L modules berlin.scm
guix system: warning: the following accounts appear more than once: zabbix
guix system: warning: the following groups appear more than once: zabbix
guix system: error: canonicalize-path: No such file or directory: "../../sync-disarchive-db.scm"
--8<---------------cut here---------------end--------------->8---

I'm positive this is new behavior.

Thanks,

Maxim

Message #21 received at 54003 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Cc: 54003 <at> debbugs.gnu.org, Aleksandr Vityazev <avityazev <at> posteo.org>
Subject: Re: bug#54003: local-file after Guile update
Date: Tue, 15 Feb 2022 20:42:21 +0100

[Message part 1 (text/plain, inline)]

Hi!

For now, here’s a workaround: instead of writing

  #~(… #$(local-file …) …)

you can write:

  (gexp (… (ungexp (local-file …)) …))

or:

  (let ((file (local-file …)))
    #~(… #$file …))

The problem is that ‘read’ hash extensions (like those for #~ and #$)
would so far return sexps with associated source properties, but psyntax
in 3.0.8 ignores those source properties.

I believe this is due to Guile commit
54bbe0b2846c5b1aa366c91d679ba724869c8cda, specifically this bit:

[Message part 2 (text/x-patch, inline)]

     (define source-annotation
       (lambda (x)
-        (if (syntax? x)
-            (syntax-sourcev x)
-            (datum-sourcev x))))
+        (and (syntax? x)
+             (syntax-sourcev x))))

[Message part 3 (text/plain, inline)]

I’m looking for a reasonable workaround we could apply.

Ludo’.

Message #24 received at 54003 <at> debbugs.gnu.org (full text, mbox):

From: Maxime Devos <maximedevos <at> telenet.be>
To: Ludovic Courtès <ludo <at> gnu.org>, Maxim Cournoyer
 <maxim.cournoyer <at> gmail.com>
Cc: 54003 <at> debbugs.gnu.org, Aleksandr Vityazev <avityazev <at> posteo.org>
Subject: Re: bug#54003: local-file after Guile update
Date: Tue, 15 Feb 2022 21:56:09 +0100

[Message part 1 (text/plain, inline)]

Ludovic Courtès schreef op di 15-02-2022 om 20:42 [+0100]:
> The problem is that ‘read’ hash extensions (like those for #~ and #$)
> would so far return sexps with associated source properties, but
> psyntax
> in 3.0.8 ignores those source properties.

Read hash extensions can return syntax objects.  In a Guile 3.0.8 REPL:

(define-module (hat))
(define (hat s-exp)
  `#(hat ,s-exp))

(read-hash-extend #\^
  (lambda (_ port)
    ;; Use 'read-syntax' instead of a combination of 'read' and
    ;; 'datum->syntax' for source properties.
    #`(hat '#,(read-syntax port))))

#^(foo bar)
$4 = #(hat (foo bar))

Doing something like this in the hash extension for G-exps might work.

Greetings,
Maxime.

[signature.asc (application/pgp-signature, inline)]

Message #27 received at 54003 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Maxime Devos <maximedevos <at> telenet.be>
Cc: 54003 <at> debbugs.gnu.org, Maxim Cournoyer <maxim.cournoyer <at> gmail.com>,
 Aleksandr Vityazev <avityazev <at> posteo.org>
Subject: Re: bug#54003: local-file after Guile update
Date: Tue, 15 Feb 2022 23:49:01 +0100

Hi,

Maxime Devos <maximedevos <at> telenet.be> skribis:

> Read hash extensions can return syntax objects.  In a Guile 3.0.8 REPL:
>
> (define-module (hat))
> (define (hat s-exp)
>   `#(hat ,s-exp))
>
> (read-hash-extend #\^
>   (lambda (_ port)
>     ;; Use 'read-syntax' instead of a combination of 'read' and
>     ;; 'datum->syntax' for source properties.
>     #`(hat '#,(read-syntax port))))
>
> #^(foo bar)
> $4 = #(hat (foo bar))
>
> Doing something like this in the hash extension for G-exps might work.

Yes, that’s one possible workaround, but then ‘read’ would find itself
returning syntax objects as well.

We have to see to what extent this is a problem in Guix proper; I think
it will at least break ‘guix style’ and related code.

Ludo’.

Message #30 received at 54003 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Maxime Devos <maximedevos <at> telenet.be>
Cc: 54003 <at> debbugs.gnu.org, Maxim Cournoyer <maxim.cournoyer <at> gmail.com>,
 Aleksandr Vityazev <avityazev <at> posteo.org>
Subject: Re: bug#54003: local-file after Guile update
Date: Wed, 16 Feb 2022 11:40:09 +0100

[Message part 1 (text/plain, inline)]

Hi!

This *ahem* pretty hack works around the problem.  I have yet to see if
it works with earlier Guile versions.  Thoughts?

I’d have preferred to avoid monkey-patching but I couldn’t think of a
way to do that.

Ludo’.

[0001-gexp-Preserve-source-location-for-and-read-extension.patch (text/x-patch, inline)]

From 7fd25c6f15f74fb6e45fc3f0db21a110267f262c Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo <at> gnu.org>
Date: Wed, 16 Feb 2022 11:27:43 +0100
Subject: [PATCH] gexp: Preserve source location for #~ and #$ read extensions.

Read hash extensions preserve source location info as source properties
on their result.  However, in Guile 3.0.8, that location would be
dismissed, leading 'local-file' to fail to resolve file names relative
to the source directory.

Fixes <https://issues.guix.gnu.org/54003>.
Reported by Aleksandr Vityazev <avityazev <at> posteo.org>.

* guix/gexp.scm <eval-when> [read-syntax-redefined?, read-procedure]
[read-syntax*]: New variables.
[read-ungexp]: Adjust to expect either sexps or syntax objects.
[read-gexp]: Call 'read-procedure'.
* tests/gexp.scm ("local-file, relative file name, within gexp")
("local-file, relative file name, within gexp, compiled"): New tests.
---
 guix/gexp.scm  | 50 +++++++++++++++++++++++++++++++++++++++++++++-----
 tests/gexp.scm | 27 +++++++++++++++++++++++++++
 2 files changed, 72 insertions(+), 5 deletions(-)

diff --git a/guix/gexp.scm b/guix/gexp.scm
index d23683e2a6..e229c1fc8f 100644
--- a/guix/gexp.scm
+++ b/guix/gexp.scm
@@ -2176,6 +2176,29 @@ (define log-port
 ;;;
 
 (eval-when (expand load eval)
+  (define-once read-syntax-redefined?
+    ;; Have we already redefined 'read-syntax'?  This needs to be done on
+    ;; 3.0.8 only to work around <https://issues.guix.gnu.org/54003>.
+    (or (not (module-variable the-scm-module 'read-syntax))
+        (not (guile-version>? "3.0.7"))))
+
+  (define read-procedure
+    ;; The current read procedure being called: either 'read' or
+    ;; 'read-syntax'.
+    (make-parameter read))
+
+  (define read-syntax*
+    ;; Replacement for 'read-syntax'.
+    (let ((read-syntax (and=> (module-variable the-scm-module 'read-syntax)
+                              variable-ref)))
+      (lambda (port . rest)
+        (parameterize ((read-procedure read-syntax))
+          (apply read-syntax port rest)))))
+
+  (unless read-syntax-redefined?
+    (set! (@ (guile) read-syntax) read-syntax*)
+    (set! read-syntax-redefined? #t))
+
   (define* (read-ungexp chr port #:optional native?)
     "Read an 'ungexp' or 'ungexp-splicing' form from PORT.  When NATIVE? is
 true, use 'ungexp-native' and 'ungexp-native-splicing' instead."
@@ -2191,22 +2214,39 @@ (define unquote-symbol
              'ungexp-native
              'ungexp))))
 
-    (match (read port)
-      ((? symbol? symbol)
-       (let ((str (symbol->string symbol)))
+    (define symbolic?
+      ;; Depending on whether (read-procedure) is 'read' or 'read-syntax', we
+      ;; might get either sexps or syntax objects.  Adjust accordingly.
+      (if (eq? (read-procedure) read)
+          symbol?
+          (compose symbol? syntax->datum)))
+
+    (define symbolic->string
+      (if (eq? (read-procedure) read)
+          symbol->string
+          (compose symbol->string syntax->datum)))
+
+    (define wrapped-symbol
+      (if (eq? (read-procedure) read)
+          (lambda (_ symbol) symbol)
+          datum->syntax))
+
+    (match ((read-procedure) port)
+      ((? symbolic? symbol)
+       (let ((str (symbolic->string symbol)))
          (match (string-index-right str #\:)
            (#f
             `(,unquote-symbol ,symbol))
            (colon
             (let ((name   (string->symbol (substring str 0 colon)))
                   (output (substring str (+ colon 1))))
-              `(,unquote-symbol ,name ,output))))))
+              `(,unquote-symbol ,(wrapped-symbol symbol name) ,output))))))
       (x
        `(,unquote-symbol ,x))))
 
   (define (read-gexp chr port)
     "Read a 'gexp' form from PORT."
-    `(gexp ,(read port)))
+    `(gexp ,((read-procedure) port)))
 
   ;; Extend the reader
   (read-hash-extend #\~ read-gexp)
diff --git a/tests/gexp.scm b/tests/gexp.scm
index bcda516623..33c0e4bf8c 100644
--- a/tests/gexp.scm
+++ b/tests/gexp.scm
@@ -28,6 +28,7 @@ (define-module (test-gexp)
   #:use-module (guix tests)
   #:use-module ((guix build utils) #:select (with-directory-excursion))
   #:use-module ((guix utils) #:select (call-with-temporary-directory))
+  #:use-module ((guix ui) #:select (load*))
   #:use-module (gnu packages)
   #:use-module (gnu packages base)
   #:use-module (gnu packages bootstrap)
@@ -222,6 +223,32 @@ (define defmod 'define-module) ;fool Geiser
       (let ((file (local-file (string-copy "../base32.scm"))))
         (local-file-absolute-file-name file)))))
 
+(test-assert "local-file, relative file name, within gexp"
+  (let* ((file     (search-path %load-path "guix/base32.scm"))
+         (interned (add-to-store %store "base32.scm" #f "sha256" file)))
+    (equal? `(the file is ,interned)
+            (gexp->sexp*
+             #~(the file is #$(local-file "../guix/base32.scm"))))))
+
+(test-assert "local-file, relative file name, within gexp, compiled"
+  ;; In Guile 3.0.8, everything read by the #~ and #$ read hash extensions
+  ;; would lack source location info, which in turn would lead
+  ;; (current-source-directory), called by 'local-file', to return #f, thereby
+  ;; breaking 'local-file' resolution.  See
+  ;; <https://issues.guix.gnu.org/54003>.
+  (let ((file (tmpnam)))
+    (call-with-output-file file
+      (lambda (port)
+        (display (string-append "#~(this file is #$(local-file \""
+                                (basename file) "\" \"t.scm\"))")
+                 port)))
+
+    (let* ((interned (add-to-store %store "t.scm" #f "sha256" file))
+           (module   (make-fresh-user-module)))
+      (module-use! module (resolve-interface '(guix gexp)))
+      (equal? `(this file is ,interned)
+              (gexp->sexp* (load* file module))))))
+
 (test-assertm "local-file, #:select?"
   (mlet* %store-monad ((select? -> (lambda (file stat)
                                      (member (basename file)

base-commit: 791069737c8c51582cc021438dae32eb0fb7b8e0
-- 
2.34.0

Message #35 received at 54003 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Maxime Devos <maximedevos <at> telenet.be>
Cc: 54003 <at> debbugs.gnu.org, Maxim Cournoyer <maxim.cournoyer <at> gmail.com>,
 Aleksandr Vityazev <avityazev <at> posteo.org>
Subject: Re: bug#54003: local-file after Guile update
Date: Wed, 16 Feb 2022 17:00:30 +0100

Ludovic Courtès <ludo <at> gnu.org> skribis:

>>From 7fd25c6f15f74fb6e45fc3f0db21a110267f262c Mon Sep 17 00:00:00 2001
> From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo <at> gnu.org>
> Date: Wed, 16 Feb 2022 11:27:43 +0100
> Subject: [PATCH] gexp: Preserve source location for #~ and #$ read extensions.
>
> Read hash extensions preserve source location info as source properties
> on their result.  However, in Guile 3.0.8, that location would be
> dismissed, leading 'local-file' to fail to resolve file names relative
> to the source directory.
>
> Fixes <https://issues.guix.gnu.org/54003>.
> Reported by Aleksandr Vityazev <avityazev <at> posteo.org>.
>
> * guix/gexp.scm <eval-when> [read-syntax-redefined?, read-procedure]
> [read-syntax*]: New variables.
> [read-ungexp]: Adjust to expect either sexps or syntax objects.
> [read-gexp]: Call 'read-procedure'.
> * tests/gexp.scm ("local-file, relative file name, within gexp")
> ("local-file, relative file name, within gexp, compiled"): New tests.

Pushed as ca155a20aea25003b03ef5e0420c77e416d5f425 after verifying that
it works with 3.0.7.

I’m leaving the bug open so we can see what to do on the Guile side.

Ludo’.

Message #40 received at 54003-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Maxime Devos <maximedevos <at> telenet.be>
Cc: Aleksandr Vityazev <avityazev <at> posteo.org>,
 Maxim Cournoyer <maxim.cournoyer <at> gmail.com>, 54003-done <at> debbugs.gnu.org
Subject: Re: bug#54003: 'local-file' fails to resolve relative file names
 with Guile 3.0.8
Date: Mon, 07 Mar 2022 14:18:58 +0100

Ludovic Courtès <ludo <at> gnu.org> skribis:

> Ludovic Courtès <ludo <at> gnu.org> skribis:
>
>>>>From 7fd25c6f15f74fb6e45fc3f0db21a110267f262c Mon Sep 17 00:00:00 2001
>> From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo <at> gnu.org>
>> Date: Wed, 16 Feb 2022 11:27:43 +0100
>> Subject: [PATCH] gexp: Preserve source location for #~ and #$ read extensions.
>>
>> Read hash extensions preserve source location info as source properties
>> on their result.  However, in Guile 3.0.8, that location would be
>> dismissed, leading 'local-file' to fail to resolve file names relative
>> to the source directory.
>>
>> Fixes <https://issues.guix.gnu.org/54003>.
>> Reported by Aleksandr Vityazev <avityazev <at> posteo.org>.
>>
>> * guix/gexp.scm <eval-when> [read-syntax-redefined?, read-procedure]
>> [read-syntax*]: New variables.
>> [read-ungexp]: Adjust to expect either sexps or syntax objects.
>> [read-gexp]: Call 'read-procedure'.
>> * tests/gexp.scm ("local-file, relative file name, within gexp")
>> ("local-file, relative file name, within gexp, compiled"): New tests.
>
> Pushed as ca155a20aea25003b03ef5e0420c77e416d5f425 after verifying that
> it works with 3.0.7.
>
> I’m leaving the bug open so we can see what to do on the Guile side.

Fixed in Guile:

  https://git.savannah.gnu.org/cgit/guile.git/commit/?id=347321ece9fc85ddf74af3c798230b7b187fbce9

Closing!

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.