GNU bug report logs - #30505
marionette/virtio-console issues lead to test failures

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 30505 in the body.
You can then email your comments to 30505 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: bug-guix <at> gnu.org
Subject: marionette/virtio-console issues lead to test failures
Date: Sun, 18 Feb 2018 01:01:31 +0100

[Message part 1 (text/plain, inline)]

Hello,

On core-updates (or is it master now?), “make check-system TESTS=basic”
fails with:

--8<---------------cut here---------------start------------->8---
Test begin:
  test-name: "/run/current-system is a GC root"
  source-file: "/gnu/store/irm2375f5p7nzyqqllwjfaby3ywhi5wq-basic-builder"
  source-line: 2
  source-form: (test-eq "/run/current-system is a GC root" (quote success!) (marionette-eval (quote (begin (add-to-load-path "/gnu/store/pn333fnrdazadmzvkbyzby8cfr176yrh-guix-0.14.0-8.bc880f9/share/guile/site/2.2") (use-modules (srfi srfi-34) (guix store)) (let ((system (readlink "/run/current-system"))) (guard (c ((nix-protocol-error? c) (and (file-exists? system) (quote success!)))) (with-store store (delete-paths store (list system)) #f))))) marionette))
Test end:
  result-kind: fail
  actual-value: #{\x1b;%G\x1b;%G\x1b;%G\x1b;%G\x1b;%G\x1b;%Gsuccess!}#
  expected-value: success!
--8<---------------cut here---------------end--------------->8---

We see similar issues with other system tests.

The “\x1b;%G” sequences correspond to the “select UTF-8” console code
(see console_codes(4)).  We’re receiving this as if we were a console,
but in fact all we want is to exchange raw bytes between the host and
the guest; we don’t want to be a full-fledged console.

I thought that using virtserialport instead of virtio-console might help
(see patch below), but the problem persists.  I’m not sure if it’s the
kernel that decides to send these codes or what.  Also unclear as to why
this happens on ‘core-updates’ and not ‘master’.

What am I missing?

Ludo’.

[Message part 2 (text/x-patch, inline)]

diff --git a/gnu/build/marionette.scm b/gnu/build/marionette.scm
index 7554a710a..173a67cef 100644
--- a/gnu/build/marionette.scm
+++ b/gnu/build/marionette.scm
@@ -1,5 +1,5 @@
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2016, 2017 Ludovic Courtès <ludo <at> gnu.org>
+;;; Copyright © 2016, 2017, 2018 Ludovic Courtès <ludo <at> gnu.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -97,8 +97,11 @@ QEMU monitor and to the guest's backdoor REPL."
           "-monitor" (string-append "unix:" socket-directory "/monitor")
           "-chardev" (string-append "socket,id=repl,path=" socket-directory
                                     "/repl")
+
+          ;; See
+          ;; <http://www.linux-kvm.org/page/VMchannel_Requirements#Invocation>.
           "-device" "virtio-serial"
-          "-device" "virtconsole,chardev=repl"))
+          "-device" "virtserialport,chardev=repl,name=org.gnu.guix.port.0"))
 
   (define (accept* port)
     (match (select (list port) '() (list port) timeout)
diff --git a/gnu/tests.scm b/gnu/tests.scm
index 3e4c3d4e3..31249f0be 100644
--- a/gnu/tests.scm
+++ b/gnu/tests.scm
@@ -1,5 +1,5 @@
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2016, 2017 Ludovic Courtès <ludo <at> gnu.org>
+;;; Copyright © 2016, 2017, 2018 Ludovic Courtès <ludo <at> gnu.org>
 ;;; Copyright © 2017 Mathieu Othacehe <m.othacehe <at> gmail.com>
 ;;; Copyright © 2017 Tobias Geerinckx-Rice <me <at> tobias.gr>
 ;;;
@@ -69,7 +69,7 @@
   marionette-configuration make-marionette-configuration
   marionette-configuration?
   (device           marionette-configuration-device ;string
-                    (default "/dev/hvc0"))
+                    (default "/dev/virtio-ports/org.gnu.guix.port.0"))
   (imported-modules marionette-configuration-imported-modules
                     (default '()))
   (requirements     marionette-configuration-requirements ;list of symbols
@@ -87,17 +87,10 @@
 
             (modules '((ice-9 match)
                        (srfi srfi-9 gnu)
-                       (guix build syscalls)
                        (rnrs bytevectors)))
             (start
-             (with-imported-modules `((guix build syscalls)
-                                      ,@imported-modules)
+             (with-imported-modules imported-modules
                #~(lambda ()
-                   (define (clear-echo termios)
-                     (set-field termios (termios-local-flags)
-                                (logand (lognot (local-flags ECHO))
-                                        (termios-local-flags termios))))
-
                    (define (self-quoting? x)
                      (letrec-syntax ((one-of (syntax-rules ()
                                                ((_) #f)
@@ -112,20 +105,7 @@
                       (dynamic-wind
                         (const #t)
                         (lambda ()
-                          (let* ((repl    (open-file #$device "r+0"))
-                                 (termios (tcgetattr (fileno repl)))
-                                 (console (open-file "/dev/console" "r+0")))
-                            ;; Don't echo input back.
-                            (tcsetattr (fileno repl) (tcsetattr-action TCSANOW)
-                                       (clear-echo termios))
-
-                            ;; Redirect output to the console.
-                            (close-fdes 1)
-                            (close-fdes 2)
-                            (dup2 (fileno console) 1)
-                            (dup2 (fileno console) 2)
-                            (close-port console)
-
+                          (let ((repl (open-file #$device "r+0")))
                             (display 'ready repl)
                             (let loop ()
                               (newline repl)
diff --git a/gnu/tests/base.scm b/gnu/tests/base.scm
index 1bc7a7027..64332000a 100644
--- a/gnu/tests/base.scm
+++ b/gnu/tests/base.scm
@@ -323,11 +323,6 @@ info --version")
             'success!
             (marionette-eval '(begin
                                 ;; Make sure the (guix …) modules are found.
-                                ;;
-                                ;; XXX: Currently shepherd and marionette run
-                                ;; on Guile 2.0 whereas Guix is on 2.2.  Yet
-                                ;; we should be able to load the 2.0 Scheme
-                                ;; files since it's pure Scheme.
                                 (add-to-load-path
                                  #+(file-append guix "/share/guile/site/2.2"))
 
@@ -337,9 +332,12 @@ info --version")
                                   (guard (c ((nix-protocol-error? c)
                                              (and (file-exists? system)
                                                   'success!)))
-                                    (with-store store
-                                      (delete-paths store (list system))
-                                      #f))))
+                                    (parameterize ((current-build-output-port
+                                                    (open-file "/dev/console"
+                                                               "r+0")))
+                                     (with-store store
+                                       (delete-paths store (list system))
+                                       #f)))))
                              marionette))
 
           ;; This symlink is currently unused, but better have it point to the

Message #8 received at 30505 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 30505 <at> debbugs.gnu.org
Subject: Re: bug#30505: marionette/virtio-console issues lead to test failures
Date: Sun, 18 Feb 2018 11:48:33 +0100

Hi Ludo,

On Sun, 18 Feb 2018 01:01:31 +0100
ludo <at> gnu.org (Ludovic Courtès) wrote:

> The “\x1b;%G” sequences correspond to the “select UTF-8” console code
> (see console_codes(4)).  We’re receiving this as if we were a console,
> but in fact all we want is to exchange raw bytes between the host and
> the guest; we don’t want to be a full-fledged console.

A lot of the tests pass console=... and thus in fact have a real console.

It might be that our new automatic console getty interferes.
I doubt it - I checked agetty sources and it doesn't touch iutf8.
Might still be worth a try to remove agetty from %base-services.

check-system TESTS=basic works fine in master indeed ...

> this happens on ‘core-updates’ and not ‘master’.

That's a good question.

Anyway, I think these codes are emitted by unicode_start - so
as a first step, sabotage the kbd package so that it can't
use unicode_start.  Does it work then?

Message #11 received at 30505 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 30505 <at> debbugs.gnu.org
Subject: Re: bug#30505: marionette/virtio-console issues lead to test failures
Date: Mon, 19 Feb 2018 16:54:44 +0100

Heya,

Danny Milosavljevic <dannym <at> scratchpost.org> skribis:

> On Sun, 18 Feb 2018 01:01:31 +0100
> ludo <at> gnu.org (Ludovic Courtès) wrote:
>
>> The “\x1b;%G” sequences correspond to the “select UTF-8” console code
>> (see console_codes(4)).  We’re receiving this as if we were a console,
>> but in fact all we want is to exchange raw bytes between the host and
>> the guest; we don’t want to be a full-fledged console.
>
> A lot of the tests pass console=... and thus in fact have a real console.
>
> It might be that our new automatic console getty interferes.
> I doubt it - I checked agetty sources and it doesn't touch iutf8.
> Might still be worth a try to remove agetty from %base-services.

This isn’t the culprit.

I hadn’t noticed this is now part of ‘%base-services’.  It would be nice
if it were enabled on ARM only.  Thoughts?

>> this happens on ‘core-updates’ and not ‘master’.
>
> That's a good question.
>
> Anyway, I think these codes are emitted by unicode_start - so
> as a first step, sabotage the kbd package so that it can't
> use unicode_start.  Does it work then?

Commenting out (display "\x1b%G" (fdes->outport fd)) in (gnu services
base) appear to solve the problem.  It seems that it used to affect just
the terminal behind FD and now somehow broadcasts to all existing
terminals?

Anyway, I’m unsure this ‘display’ call was needed at all.  It seems
redundant with the ‘tcsetattr’ call below.  So I think we’ll just remove
it.

Thoughts?

Ludo’.

Message #14 received at 30505 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 30505 <at> debbugs.gnu.org
Subject: Re: bug#30505: marionette/virtio-console issues lead to test failures
Date: Mon, 19 Feb 2018 18:08:49 +0100

Hi Ludo,

>I hadn’t noticed this is now part of ‘%base-services’.  It would be nice
if it were enabled on ARM only.  Thoughts?

Why?  It's not ARM-specific and there are people using headless x86 servers
posting on the mailing list :)

It's only enabled when you specify a serial port as console on the Linux
command line - that's not going to happen accidentially.

And once Linux uses the console for its messages it's nice to also have a
login process running in the end - otherwise it's kinda annoying having
only a read-only line when you sit right in front of the machine.

On Mon, 19 Feb 2018 16:54:44 +0100
ludo <at> gnu.org (Ludovic Courtès) wrote:

> Commenting out (display "\x1b%G" (fdes->outport fd)) in (gnu services
> base) appear to solve the problem.  It seems that it used to affect just
> the terminal behind FD and now somehow broadcasts to all existing
> terminals?

It was a bad idea to do the "\x1b%G" in the first place.

There's a Linux kernel command-line parameter "vt.default_utf8" which
is set to true anyway.  In that case the iflag IUTF8 is set automatically
by Linux drivers/tty/vt/vt.c and the driver also does the same as "\x1b%G"
does in that case.

So what do these things in (gnu services base) accomplish?  Sounds like
they change nothing.

Maybe that was only done in later Linux kernels? I checked 3.4.103, it did that
already.

Message #17 received at 30505 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 30505 <at> debbugs.gnu.org
Subject: Re: bug#30505: marionette/virtio-console issues lead to test failures
Date: Mon, 19 Feb 2018 21:35:05 +0100

Hello,

Danny Milosavljevic <dannym <at> scratchpost.org> skribis:

>>I hadn’t noticed this is now part of ‘%base-services’.  It would be nice
> if it were enabled on ARM only.  Thoughts?
>
> Why?  It's not ARM-specific and there are people using headless x86 servers
> posting on the mailing list :)
>
> It's only enabled when you specify a serial port as console on the Linux
> command line - that's not going to happen accidentially.
>
> And once Linux uses the console for its messages it's nice to also have a
> login process running in the end - otherwise it's kinda annoying having
> only a read-only line when you sit right in front of the machine.

Oh OK, got it, that makes sense.  :-)

> On Mon, 19 Feb 2018 16:54:44 +0100
> ludo <at> gnu.org (Ludovic Courtès) wrote:
>
>> Commenting out (display "\x1b%G" (fdes->outport fd)) in (gnu services
>> base) appear to solve the problem.  It seems that it used to affect just
>> the terminal behind FD and now somehow broadcasts to all existing
>> terminals?
>
> It was a bad idea to do the "\x1b%G" in the first place.

Because it’s redundant with IUTF8?

> There's a Linux kernel command-line parameter "vt.default_utf8" which
> is set to true anyway.  In that case the iflag IUTF8 is set automatically
> by Linux drivers/tty/vt/vt.c and the driver also does the same as "\x1b%G"
> does in that case.
>
> So what do these things in (gnu services base) accomplish?  Sounds like
> they change nothing.
>
> Maybe that was only done in later Linux kernels? I checked 3.4.103, it did that
> already.

This ‘unicode-start’ procedure is essentially a port of the
‘unicode_start’ script from ‘kbd’.  I suppose the justification is to
make sure we’re using UTF-8 input regardless of what the kernel defaults
or command-line options are.

Thoughts?

Ludo’.

Message #22 received at 30505-done <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 30505-done <at> debbugs.gnu.org
Subject: Re: bug#30505: marionette/virtio-console issues lead to test failures
Date: Mon, 19 Feb 2018 22:53:30 +0100

Danny Milosavljevic <dannym <at> scratchpost.org> skribis:

> On Mon, 19 Feb 2018 16:54:44 +0100
> ludo <at> gnu.org (Ludovic Courtès) wrote:
>
>> Commenting out (display "\x1b%G" (fdes->outport fd)) in (gnu services
>> base) appear to solve the problem.  It seems that it used to affect just
>> the terminal behind FD and now somehow broadcasts to all existing
>> terminals?
>
> It was a bad idea to do the "\x1b%G" in the first place.

Commit ce0a62f6c5c8ea486869360f654356777cdf918e gets rid of that.

I’ve also pushed the switch to “VMchannel” as commit
27a2c9c3e071fdb380c2f4f389b7cf4008dc75f7.

Thanks,
Ludo’.

Message #25 received at 30505 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 30505 <at> debbugs.gnu.org
Subject: Re: bug#30505: marionette/virtio-console issues lead to test failures
Date: Mon, 19 Feb 2018 23:35:53 +0100

Hi Ludo,

On Mon, 19 Feb 2018 21:35:05 +0100
ludo <at> gnu.org (Ludovic Courtès) wrote:

> > It was a bad idea to do the "\x1b%G" in the first place.  
> 
> Because it’s redundant with IUTF8?

I meant because the Linux kernel does it already and it's better not to
have random multi-byte racy writes onto the tty while the mingetty
is starting up (and possibly buffering and pending half of another sequence).

As far as I understand it's not redundant to do both.

According to src/Linux/linux-4.12-rc2/drivers/tty/vt/vt.c, the 'G' controls
the conversion utf-8->unicode that happens before the virtual terminal
displays a corresponding character on the screen.

On the other hand, the termios iutf8 is meant for the program running in the session.

Say you have bash on vt1, then bash can check termios for the settings and find
out whether vt1 is UTF-8-capable (also has other settings like whether the
terminal already supports line editing etc - old-school terminals were quite
cool; a friend of mine salvaged a real one ^^).

Note that drivers/tty/vt/vt.c only copies ONE way, from the 'G' flag to the
termios (and that seldomly).
Makes sense since the programs shouldn't have a say in what the terminal can do :)

So I'd say guix services fiddling with termios is ... weird and the 'G' slightly
less weird.

> This ‘unicode-start’ procedure is essentially a port of the
> ‘unicode_start’ script from ‘kbd’.  I suppose the justification is to
> make sure we’re using UTF-8 input regardless of what the kernel defaults
> or command-line options are.

Yeah, but it's asking for trouble.

I just checked Linux 2.6.32.1, it defaults to utf8 (IUTF8 in termios, and 'G').

I'd suggest to remove both.

Message #28 received at 30505 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 30505 <at> debbugs.gnu.org
Subject: Starting console/terminal Unicode support
Date: Wed, 21 Feb 2018 23:21:34 +0100

Hi Danny,

Danny Milosavljevic <dannym <at> scratchpost.org> skribis:

> On Mon, 19 Feb 2018 21:35:05 +0100
> ludo <at> gnu.org (Ludovic Courtès) wrote:
>
>> > It was a bad idea to do the "\x1b%G" in the first place.  
>> 
>> Because it’s redundant with IUTF8?
>
> I meant because the Linux kernel does it already and it's better not to
> have random multi-byte racy writes onto the tty while the mingetty
> is starting up (and possibly buffering and pending half of another sequence).

That ‘unicode-start’ code run when the ‘console-font-ttyN’ service is
started, right after ‘term-ttyN’ (mingetty) is up.  You’d hope mingetty
is really up and running once ‘term-ttyN’ is up, no?

> As far as I understand it's not redundant to do both.
>
> According to src/Linux/linux-4.12-rc2/drivers/tty/vt/vt.c, the 'G' controls
> the conversion utf-8->unicode that happens before the virtual terminal
> displays a corresponding character on the screen.
>
> On the other hand, the termios iutf8 is meant for the program running in the session.

OK, I see.

> Say you have bash on vt1, then bash can check termios for the settings and find
> out whether vt1 is UTF-8-capable (also has other settings like whether the
> terminal already supports line editing etc - old-school terminals were quite
> cool; a friend of mine salvaged a real one ^^).
>
> Note that drivers/tty/vt/vt.c only copies ONE way, from the 'G' flag to the
> termios (and that seldomly).
> Makes sense since the programs shouldn't have a say in what the terminal can do :)
>
> So I'd say guix services fiddling with termios is ... weird and the 'G' slightly
> less weird.

[...]

>> This ‘unicode-start’ procedure is essentially a port of the
>> ‘unicode_start’ script from ‘kbd’.  I suppose the justification is to
>> make sure we’re using UTF-8 input regardless of what the kernel defaults
>> or command-line options are.
>
> Yeah, but it's asking for trouble.
>
> I just checked Linux 2.6.32.1, it defaults to utf8 (IUTF8 in termios, and 'G').
>
> I'd suggest to remove both.

When we’re using the defaults, I understand both are unnecessary.  In
other cases (custom kernel build with different defaults, different
kernel command-line settings, etc.), we’d just get it wrong with things
breaking down the path, no?

Also, FWIW, systemd’s vconsole-setup.c does exactly that:

  https://github.com/systemd/systemd/blob/master/src/vconsole/vconsole-setup.c#L94

And again that corresponds to the ‘unicode_start’ script.

Thoughts?

Ludo’.

Message #31 received at 30505 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 30505 <at> debbugs.gnu.org
Subject: Re: Starting console/terminal Unicode support
Date: Thu, 22 Feb 2018 00:01:05 +0100

Hi Ludo,

On Wed, 21 Feb 2018 23:21:34 +0100
ludo <at> gnu.org (Ludovic Courtès) wrote:

> >> This ‘unicode-start’ procedure is essentially a port of the
> >> ‘unicode_start’ script from ‘kbd’.  I suppose the justification is to
> >> make sure we’re using UTF-8 input regardless of what the kernel defaults
> >> or command-line options are.  

Yeah, but the only way to have it not use utf-8 would be to explicitly pass
"default_utf8=0" as a command-line parameter.  I think at that point the
user deserves what he personally asked for :P

>In
> other cases (custom kernel build with different defaults, different
> kernel command-line settings, etc.), we’d just get it wrong with things
> breaking down the path, no?

Yes, it would just default to ASCII.  In the big scheme of things it's not so bad.

I just thought that if it causes trouble *and* it's not needed anymore, the
simplest way is to just remove it.

> Also, FWIW, systemd’s vconsole-setup.c does exactly that:
> 
>   https://github.com/systemd/systemd/blob/master/src/vconsole/vconsole-setup.c#L94

Huh, I wonder why they do it.  systemd is quite new.  I understand that older stuff
(like kbd) did it because they had to in the Linux 2.4 days - but this?

We should ask them...

Aha, they also have

static int toggle_utf8_sysfs(bool utf8) {
        int r;

        r = write_string_file("/sys/module/vt/parameters/default_utf8", one_zero(utf8), 0);
        if (r < 0)
                return log_warning_errno(r, "Failed to %s sysfs UTF-8 flag: %m", enable_disable(utf8));

        log_debug("Sysfs UTF-8 flag %sd", enable_disable(utf8));
        return 0;
}

so we could in fact directly force the VT driver to do our bidding, or even
check what's up before trying to write the 'G' (the latter sounds like a hack
hiding the actual problem, though).

Message #34 received at 30505 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 30505 <at> debbugs.gnu.org
Subject: Re: Starting console/terminal Unicode support
Date: Fri, 23 Feb 2018 22:41:19 +0100

Hello!

Danny Milosavljevic <dannym <at> scratchpost.org> skribis:

> On Wed, 21 Feb 2018 23:21:34 +0100
> ludo <at> gnu.org (Ludovic Courtès) wrote:
>
>> >> This ‘unicode-start’ procedure is essentially a port of the
>> >> ‘unicode_start’ script from ‘kbd’.  I suppose the justification is to
>> >> make sure we’re using UTF-8 input regardless of what the kernel defaults
>> >> or command-line options are.  
>
> Yeah, but the only way to have it not use utf-8 would be to explicitly pass
> "default_utf8=0" as a command-line parameter.  I think at that point the
> user deserves what he personally asked for :P

(He or she.)  But yeah, that makes sense.

>>In
>> other cases (custom kernel build with different defaults, different
>> kernel command-line settings, etc.), we’d just get it wrong with things
>> breaking down the path, no?
>
> Yes, it would just default to ASCII.  In the big scheme of things it's not so bad.
>
> I just thought that if it causes trouble *and* it's not needed anymore, the
> simplest way is to just remove it.

Yeah.

>> Also, FWIW, systemd’s vconsole-setup.c does exactly that:
>> 
>>   https://github.com/systemd/systemd/blob/master/src/vconsole/vconsole-setup.c#L94
>
> Huh, I wonder why they do it.  systemd is quite new.  I understand that older stuff
> (like kbd) did it because they had to in the Linux 2.4 days - but this?
>
> We should ask them...
>
> Aha, they also have
>
> static int toggle_utf8_sysfs(bool utf8) {
>         int r;
>
>         r = write_string_file("/sys/module/vt/parameters/default_utf8", one_zero(utf8), 0);
>         if (r < 0)
>                 return log_warning_errno(r, "Failed to %s sysfs UTF-8 flag: %m", enable_disable(utf8));
>
>         log_debug("Sysfs UTF-8 flag %sd", enable_disable(utf8));
>         return 0;
> }
>
> so we could in fact directly force the VT driver to do our bidding, or even
> check what's up before trying to write the 'G' (the latter sounds like a hack
> hiding the actual problem, though).

Yeah, we could simply write “1” to that file I guess.

Let’s do that?

Thanks for explaining,
Ludo’.

Message #41 received at 30505 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 30505 <at> debbugs.gnu.org
Subject: Re: Starting console/terminal Unicode support
Date: Sun, 25 Feb 2018 13:31:31 +0100

Hi Ludo,

On Fri, 23 Feb 2018 22:41:19 +0100
ludo <at> gnu.org (Ludovic Courtès) wrote:

> Yeah, we could simply write “1” to that file I guess.
> 
> Let’s do that?

I've thought about it and since non-UTF8 terminal probably only can be found in
museums (famous last words), yeah, let's do that.

Usually I'm not a fan of overriding user settings (in this case the kernel
command line) but in this case it's OK since the rest of the system at least
can be sure that we have UTF-8 VTs.

Message #44 received at 30505 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 30505 <at> debbugs.gnu.org
Subject: Re: bug#30505: Starting console/terminal Unicode support
Date: Sun, 25 Feb 2018 13:57:04 +0100

> Usually I'm not a fan of overriding user settings (in this case the kernel
> command line) but in this case it's OK since the rest of the system at least
> can be sure that we have UTF-8 VTs.

Aha, the gnu/services/base.scm tries to make sure that an UTF-8 aware unicode
font is already loaded before switching the tty to UTF-8.  We would have to
set default_utf8 earlier (before even the vt is activated), otherwise the
setting wouldn't be copied to the new terminal.

Message #47 received at 30505 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 30505 <at> debbugs.gnu.org
Subject: Re: bug#30505: Starting console/terminal Unicode support
Date: Mon, 26 Feb 2018 18:09:53 +0100

Hey,

Danny Milosavljevic <dannym <at> scratchpost.org> skribis:

>> Usually I'm not a fan of overriding user settings (in this case the kernel
>> command line) but in this case it's OK since the rest of the system at least
>> can be sure that we have UTF-8 VTs.
>
> Aha, the gnu/services/base.scm tries to make sure that an UTF-8 aware unicode
> font is already loaded before switching the tty to UTF-8.  We would have to
> set default_utf8 earlier (before even the vt is activated), otherwise the
> setting wouldn't be copied to the new terminal.

You’re talking about %default-console-font?  What does that change?

I’m lost.  :-)

We have the following options:

  1. status quo, i.e., tcsetattr without %G in the ‘term-ttyN’ services;

  2. remove the tcsetattr call, write “1” to
     /sys/module/vt/parameters/default_utf8;

  3. do nothing (no tcsetattr, no /sys, no %G).

It seems that systemd does everything: %G, tcsetattr, and /sys.

What do we do now?

Ludo’.

Message #50 received at 30505 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 30505 <at> debbugs.gnu.org
Subject: Re: bug#30505: Starting console/terminal Unicode support
Date: Mon, 26 Feb 2018 23:23:02 +0100

Hi Ludo,

On Mon, 26 Feb 2018 18:09:53 +0100
ludo <at> gnu.org (Ludovic Courtès) wrote:

>   1. status quo, i.e., tcsetattr without %G in the ‘term-ttyN’ services;
> 
>   2. remove the tcsetattr call, write “1” to
>      /sys/module/vt/parameters/default_utf8;
> 
>   3. do nothing (no tcsetattr, no /sys, no %G).
> 
> It seems that systemd does everything: %G, tcsetattr, and /sys.

We have the following options as a distribution:

(a) Hope that the user sets up everything as he should - iff they set up 
the kernel as non-utf8, they also must set up an non-utf-8 console font.
(b) Force their hand by decreeing that GuixSD is utf-8 only.  That's what
we are doing now I think - and a lot of projects (GNOME etc) assume UTF-8
nowadays anyway.

Since we just want to fix the bug and leave everything else as it is (b),
I'd vote for your (2.).

(2.) needs to be done early before the VTs are actually there.

default_utf8 is a default, and it will be read only when defaulting, so
in this case it will be read when Linux is constructing a tty.

%base-services contain both console-font-service and mingetty-service which
both access the ttys.  Linux VTs are created on demand.

Once the VTs are created, us setting default_utf8 is too late.

We should set default_utf8 before anyone touches tty[123456].

Where will that be?

Should we just set it in boot-system ? That's probably a nice way -
also since I think this setting is quite Linux-specific.

Message #53 received at 30505 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 30505 <at> debbugs.gnu.org
Subject: Re: bug#30505: Starting console/terminal Unicode support
Date: Tue, 27 Feb 2018 10:25:36 +0100

Hi,

Danny Milosavljevic <dannym <at> scratchpost.org> skribis:

> On Mon, 26 Feb 2018 18:09:53 +0100
> ludo <at> gnu.org (Ludovic Courtès) wrote:
>
>>   1. status quo, i.e., tcsetattr without %G in the ‘term-ttyN’ services;
>> 
>>   2. remove the tcsetattr call, write “1” to
>>      /sys/module/vt/parameters/default_utf8;
>> 
>>   3. do nothing (no tcsetattr, no /sys, no %G).
>> 
>> It seems that systemd does everything: %G, tcsetattr, and /sys.
>
> We have the following options as a distribution:
>
> (a) Hope that the user sets up everything as he should - iff they set up 
> the kernel as non-utf8, they also must set up an non-utf-8 console font.
> (b) Force their hand by decreeing that GuixSD is utf-8 only.  That's what
> we are doing now I think - and a lot of projects (GNOME etc) assume UTF-8
> nowadays anyway.
>
> Since we just want to fix the bug and leave everything else as it is (b),
> I'd vote for your (2.).
>
> (2.) needs to be done early before the VTs are actually there.

OK, makes sense, I agree.

> default_utf8 is a default, and it will be read only when defaulting, so
> in this case it will be read when Linux is constructing a tty.
>
> %base-services contain both console-font-service and mingetty-service which
> both access the ttys.  Linux VTs are created on demand.
>
> Once the VTs are created, us setting default_utf8 is too late.
>
> We should set default_utf8 before anyone touches tty[123456].
>
> Where will that be?
>
> Should we just set it in boot-system ? That's probably a nice way -
> also since I think this setting is quite Linux-specific.

Alternately we could introduce a ‘virtual-terminal’ service that every
‘term-*’ service would depend on.  That ‘virtual-terminal’ service would
simply write “1” to /sys/…/default_utf8.

How does that sound?

Ludo’.

Message #58 received at 30505-done <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 30505-done <at> debbugs.gnu.org
Subject: Re: bug#30505: Starting console/terminal Unicode support
Date: Thu, 15 Mar 2018 11:41:05 +0100

Hello!

ludo <at> gnu.org (Ludovic Courtès) skribis:

> Danny Milosavljevic <dannym <at> scratchpost.org> skribis:

[...]

>> default_utf8 is a default, and it will be read only when defaulting, so
>> in this case it will be read when Linux is constructing a tty.
>>
>> %base-services contain both console-font-service and mingetty-service which
>> both access the ttys.  Linux VTs are created on demand.
>>
>> Once the VTs are created, us setting default_utf8 is too late.
>>
>> We should set default_utf8 before anyone touches tty[123456].
>>
>> Where will that be?
>>
>> Should we just set it in boot-system ? That's probably a nice way -
>> also since I think this setting is quite Linux-specific.
>
> Alternately we could introduce a ‘virtual-terminal’ service that every
> ‘term-*’ service would depend on.  That ‘virtual-terminal’ service would
> simply write “1” to /sys/…/default_utf8.

Done in bb3062ad6290223ea24144ca8aa1f4cddac8f9be.

Thanks!

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.