GNU bug report logs - #21536
Texi, Unicode and Emacs interface

Previous Next

Package: guix;

Reported by: Alex Kost <alezost <at> gmail.com>

Date: Wed, 23 Sep 2015 10:52:02 UTC

Severity: normal

Done: ludo <at> gnu.org (Ludovic Courtès)

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 21536 in the body.
You can then email your comments to 21536 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-guix <at> gnu.org:
bug#21536; Package guix. (Wed, 23 Sep 2015 10:52:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Alex Kost <alezost <at> gmail.com>:
New bug report received and forwarded. Copy sent to bug-guix <at> gnu.org. (Wed, 23 Sep 2015 10:52:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Alex Kost <alezost <at> gmail.com>
To: bug-guix <at> gnu.org
Subject: Texi, Unicode and Emacs interface
Date: Wed, 23 Sep 2015 13:51:36 +0300
[Message part 1 (text/plain, inline)]
There is an interesting "bug" that leads to a problem with Emacs
interface for Guix packages.

Try "M-x guix-search-by-name RET mpfrcx".  You will get an
'encoding-error'.  This happens because:

1. Description of the 'mpfrcx' package contains a unicode symbol "–".

2. For some reason unicode symbols are not handled properly in REPLs
   connected to a guile server.

To illustrate this problem, here is the recipe:

1. Run guile server:

  guile --listen

Evaluate there:

  ((@@ (guix ui) texi->plain-text) "foo \u2015 bar.")

So far so good.

2. Now connect to it, either with:

   - netcat: "netcat localhost 37146"

   - or Geiser: "M-x connect-to-guile"

and evaluate the same expression.  This time you will get the error.

(This should probably be counted as a Guile bug, I'm not sure)

To make the connected REPL handle such unicode strings you have to
manually evaluate (setlocale LC_ALL "") there (even (setlocale LC_ALL)
works!).  So I used this as a workaround for the emacs interface in the
attached patch (I don't know if there is a better workaround).


BTW, since we now use texi for the package descriptions, does it mean
that our intention is to get rid of using unicode symbols directly?

…Hm, I've just tried ((@@ (guix ui) texi->plain-text) "@U{2012}") and
got:

  Throw to key `parser-error' with args `(#f "Unknown command" U)

[0001-emacs-Handle-unicode-symbols-in-texi-descriptions.patch (text/x-patch, attachment)]

Information forwarded to bug-guix <at> gnu.org:
bug#21536; Package guix. (Wed, 23 Sep 2015 22:03:02 GMT) Full text and rfc822 format available.

Message #8 received at 21536 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Alex Kost <alezost <at> gmail.com>
Cc: 21536 <at> debbugs.gnu.org
Subject: Re: bug#21536: Texi, Unicode and Emacs interface
Date: Thu, 24 Sep 2015 00:02:16 +0200
[Message part 1 (text/plain, inline)]
Alex Kost <alezost <at> gmail.com> skribis:

> Evaluate there:
>
>   ((@@ (guix ui) texi->plain-text) "foo \u2015 bar.")
>
> So far so good.
>
> 2. Now connect to it, either with:
>
>    - netcat: "netcat localhost 37146"
>
>    - or Geiser: "M-x connect-to-guile"
>
> and evaluate the same expression.  This time you will get the error.

The encoding error comes from the fact that ‘texi->plain-text’ uses a
string port, and string ports internally use the current locale encoding
or ‘%default-port-encoding’.

Consequently, when running in the “C” locale, string ports cannot
represent non-ASCII code points (something widely regarded as a bug in
Guile, and at the very least an annoyance.)

To work around that, you can type this in *Guix Internal REPL*:

  (fluid-set! %default-port-encoding "UTF-8")

I fixed in commit 2cad18a8 of guix-artwork.git, but perhaps a similar
hack is apparently needed elsewhere.

Could you test this patch:

[Message part 2 (text/x-patch, inline)]
diff --git a/guix/ui.scm b/guix/ui.scm
index 4a3630f..67dd062 100644
--- a/guix/ui.scm
+++ b/guix/ui.scm
@@ -803,7 +803,10 @@ converted to a space; sequences of more than one line break are preserved."
 
 (define (texi->plain-text str)
   "Return a plain-text representation of texinfo fragment STR."
-  (stexi->plain-text (texi-fragment->stexi str)))
+  ;; 'texi-fragment->stexi' uses a string port so make sure it's a
+  ;; Unicode-capable one (see <http://bugs.gnu.org/11197>.)
+  (with-fluids ((%default-port-encoding "UTF-8"))
+    (stexi->plain-text (texi-fragment->stexi str))))
 
 (define (package-description-string package)
   "Return a plain-text representation of PACKAGE description field."
[Message part 3 (text/plain, inline)]
> BTW, since we now use texi for the package descriptions, does it mean
> that our intention is to get rid of using unicode symbols directly?

Not particularly.

Thanks,
Ludo’.

Information forwarded to bug-guix <at> gnu.org:
bug#21536; Package guix. (Thu, 24 Sep 2015 12:03:01 GMT) Full text and rfc822 format available.

Message #11 received at 21536 <at> debbugs.gnu.org (full text, mbox):

From: Alex Kost <alezost <at> gmail.com>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 21536 <at> debbugs.gnu.org
Subject: Re: bug#21536: Texi, Unicode and Emacs interface
Date: Thu, 24 Sep 2015 15:02:58 +0300
Ludovic Courtès (2015-09-24 01:02 +0300) wrote:

> Alex Kost <alezost <at> gmail.com> skribis:
>
>> Evaluate there:
>>
>>   ((@@ (guix ui) texi->plain-text) "foo \u2015 bar.")
>>
>> So far so good.
>>
>> 2. Now connect to it, either with:
>>
>>    - netcat: "netcat localhost 37146"
>>
>>    - or Geiser: "M-x connect-to-guile"
>>
>> and evaluate the same expression.  This time you will get the error.
>
> The encoding error comes from the fact that ‘texi->plain-text’ uses a
> string port, and string ports internally use the current locale encoding
> or ‘%default-port-encoding’.
>
> Consequently, when running in the “C” locale, string ports cannot
> represent non-ASCII code points (something widely regarded as a bug in
> Guile, and at the very least an annoyance.)
>
> To work around that, you can type this in *Guix Internal REPL*:
>
>   (fluid-set! %default-port-encoding "UTF-8")
>
> I fixed in commit 2cad18a8 of guix-artwork.git, but perhaps a similar
> hack is apparently needed elsewhere.

Ah, I didn't follow 'guix-artwork' repo, now I see.

> Could you test this patch:
>
>
> diff --git a/guix/ui.scm b/guix/ui.scm
> index 4a3630f..67dd062 100644
> --- a/guix/ui.scm
> +++ b/guix/ui.scm
> @@ -803,7 +803,10 @@ converted to a space; sequences of more than one line break are preserved."
>  
>  (define (texi->plain-text str)
>    "Return a plain-text representation of texinfo fragment STR."
> -  (stexi->plain-text (texi-fragment->stexi str)))
> +  ;; 'texi-fragment->stexi' uses a string port so make sure it's a
> +  ;; Unicode-capable one (see <http://bugs.gnu.org/11197>.)
> +  (with-fluids ((%default-port-encoding "UTF-8"))
> +    (stexi->plain-text (texi-fragment->stexi str))))
>  
>  (define (package-description-string package)
>    "Return a plain-text representation of PACKAGE description field."

Yes, I confirm that with this change the problem with Emacs interface is
solved, thank you!

-- 
Alex




Reply sent to ludo <at> gnu.org (Ludovic Courtès):
You have taken responsibility. (Thu, 24 Sep 2015 20:18:01 GMT) Full text and rfc822 format available.

Notification sent to Alex Kost <alezost <at> gmail.com>:
bug acknowledged by developer. (Thu, 24 Sep 2015 20:18:02 GMT) Full text and rfc822 format available.

Message #16 received at 21536-done <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Alex Kost <alezost <at> gmail.com>
Cc: 21536-done <at> debbugs.gnu.org
Subject: Re: bug#21536: Texi, Unicode and Emacs interface
Date: Thu, 24 Sep 2015 22:16:55 +0200
Alex Kost <alezost <at> gmail.com> skribis:

> Yes, I confirm that with this change the problem with Emacs interface is
> solved, thank you!

Pushed as 08d7e35, thanks!

Ludo’.




bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Fri, 23 Oct 2015 11:24:04 GMT) Full text and rfc822 format available.

This bug report was last modified 8 years and 160 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.