GNU bug report logs - #15608
kbd.scm unicode problem

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 15608 in the body.
You can then email your comments to 15608 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: "Guy Ze Grant" <gzg <at> riseup.net>
To: bug-guix <at> gnu.org
Subject: kbd.scm unicode problem
Date: Mon, 14 Oct 2013 01:14:43 -0700

[Message part 1 (text/plain, inline)]

I was asked to post here, so here it is; Attached is my expression thus
far for kbd, at the patch phase though -- the following issue pops up:
"find-files: ./doc/utf/??????: No such file or directory", along with a
spew of guile barf following (which you can see in the kbd.scm.log file).

Not sure if it's a pertaining to my lack of experience, or if it's
something actually wrong with guix itself, but might as-well through it
off to the experts. :^)

[kbd.scm (application/octet-stream, attachment)]

[kbd.scm.log (application/octet-stream, attachment)]

Message #8 received at 15608 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: "Guy Ze Grant" <gzg <at> riseup.net>
Cc: 15608 <at> debbugs.gnu.org
Subject: Re: bug#15608: kbd.scm unicode problem
Date: Wed, 16 Oct 2013 15:32:37 +0200

"Guy Ze Grant" <gzg <at> riseup.net> skribis:

> I was asked to post here, so here it is; Attached is my expression thus
> far for kbd, at the patch phase though -- the following issue pops up:
> "find-files: ./doc/utf/??????: No such file or directory", along with a
> spew of guile barf following (which you can see in the kbd.scm.log file).

As we discussed on IRC, the problem is that POSIX considers file names
to be just byte strings, and the encoding is up to the application, and
locale-dependent by default.  In practice people typically use UTF-8
(GLib and friends expect that.)

Here’s the problem and fix:

--8<---------------cut here---------------start------------->8---
$ echo $LANG
en_US.UTF-8

$ ls -a
.  ..  ♪♬

$ guile -c '(use-modules (ice-9 ftw)) (pk (scandir "."))'

;;; (("." ".." "??????"))

$ guile -c '(use-modules (ice-9 ftw)) (setlocale LC_ALL "en_US.utf8") (pk (scandir "."))'

;;; (("." ".." "♪♬"))
--8<---------------cut here---------------end--------------->8---

In the first run Guile uses the “C” locale, so it’s unable to decode “♪♬”.

In the second run it uses a UTF-8 locale, so everything works fine.

Back to kbd: can you add a phase after the ‘unpack’ phase that just
does this?

  (setlocale "LC_ALL" "en_US.utf8")

If that works we can probably make it the default in the ‘core-updates’
branch.

> (define-module (gnu packages kbd)

Since kbd is Linux-specific, could you put it in (gnu packages linux)?

[...]

>  102: 1 [patch-source-shebangs # ...]
> In unknown file:
>    ?: 0 [sort # #<procedure string<? (#:optional _ _ . _)>]
>
> ERROR: In procedure sort:
> ERROR: In procedure list-copy: Wrong type argument in position 1: ("./config/ylwrap" "./config/install-sh" "./config/depcomp" "./config/config.sub" "./config/config.rpath" "./config/missing" "./config/config.guess" "./config/mkinstalldirs" "./config/compile" "./
[...]
> " "./doc/utf/README" "./doc/utf/ethiopic" . #f)

It’s also a bug that ‘find-files’ returns a improper list when
encountering such an issue.  I’ll fit it in ‘core-updates’.

Thanks,
Ludo’.

Message #11 received at 15608 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: "Guy Ze Grant" <gzg <at> riseup.net>
Cc: 15608 <at> debbugs.gnu.org
Subject: Re: bug#15608: kbd.scm unicode problem
Date: Tue, 10 Dec 2013 22:59:57 +0100

ludo <at> gnu.org (Ludovic Courtès) skribis:

> Back to kbd: can you add a phase after the ‘unpack’ phase that just
> does this?
>
>   (setlocale "LC_ALL" "en_US.utf8")

Did you try this?

What happened to the kdb package definition?

Thanks,
Ludo’.

Message #16 received at 15608 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: "Guy Ze Grant" <gzg <at> riseup.net>
Cc: 15608 <at> debbugs.gnu.org
Subject: Re: bug#15608: kbd.scm unicode problem
Date: Fri, 28 Feb 2014 12:16:16 +0100

ludo <at> gnu.org (Ludovic Courtès) skribis:

> ludo <at> gnu.org (Ludovic Courtès) skribis:
>
>> Back to kbd: can you add a phase after the ‘unpack’ phase that just
>> does this?
>>
>>   (setlocale "LC_ALL" "en_US.utf8")
>
> Did you try this?
>
> What happened to the kdb package definition?

Ping!

Ludo’.

Message #21 received at 15608-done <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: "Guy Ze Grant" <gzg <at> riseup.net>
Cc: 15608-done <at> debbugs.gnu.org
Subject: Re: bug#15608: kbd.scm unicode problem
Date: Tue, 17 Jun 2014 14:20:36 +0200

Two things have happened:

  1. Commit f135276 fixed ‘find-files’ to gracefully handle file names
     that use an unsupported encoding.

  2. I packaged kdb (commit b10e9ff6), which works nicely.

Thanks,
Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.