GNU bug report logs - #28772
guix system reconfigure after kernel panic user or group not created

Previous Next

To reply to this bug, email your comments to 28772 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Oleg Pykhalov <go.wigust <at> gmail.com>
To: bug-guix <at> gnu.org
Subject: guix system reconfigure after kernel panic user or group not created
Date: Tue, 10 Oct 2017 08:51:15 +0300

Hello Guix,

During 'guix system reconfigure' I got a kernel panic.

Getting it quite often while 'guix system reconfigure' at Linux magnolia
4.13.4-gnu #1 SMP 1 x86_64 GNU/Linux.  But it's not a subject of current
report.

After reboot I tried another attempt to make a 'guix system
reconfigure', but because of /etc/group.lock and /etc/passwd.lock
operating-system groups and users fields didn't applied.

I tried to get a creation of adbusers group and add my user to it, but
because of those files I cannot do it, until manually removed them.

So, do we need to check if *.lock files are exist and remove them at the
start of 'guix system reconfigure' OR abort reconfigure and creation of
Grub boot entry OR something else?

Thanks,
Oleg.

Message #8 received at 28772 <at> debbugs.gnu.org (full text, mbox):

From: Oleg Pykhalov <go.wigust <at> gmail.com>
To: 28772 <at> debbugs.gnu.org
Subject: Re: bug#28772: guix system reconfigure after kernel panic user or
 group not created
Date: Tue, 10 Oct 2017 17:55:39 +0300

Oleg Pykhalov <go.wigust <at> gmail.com> writes:

[...]

> Getting it quite often while 'guix system reconfigure' at Linux magnolia
> 4.13.4-gnu #1 SMP 1 x86_64 GNU/Linux.  But it's not a subject of current
> report.

Kernel panic was caused by my miss configuring.  I used a Guile module
on reconfigure via GUILE_LOAD_PATH and (use-modles (my-module) in
config.scm.  I guess it's also required after reconfigure, according to
some times getting errors after reconfigure: cannot find my-module.  So
I need to add my-module globally on system.

[...]

Message #11 received at 28772 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Oleg Pykhalov <go.wigust <at> gmail.com>
Cc: 28772 <at> debbugs.gnu.org
Subject: Re: bug#28772: guix system reconfigure after kernel panic user or
 group not created
Date: Tue, 10 Oct 2017 17:37:39 +0200

Hello,

Oleg Pykhalov <go.wigust <at> gmail.com> skribis:

> During 'guix system reconfigure' I got a kernel panic.

Can you show the exact command and its output?

A user-land program is not supposed to be able to cause a kernel panic;
if it does, that’s a kernel bug.

But perhaps you got the kernel panic *after* rebooting in the
reconfigured system?  That could well be a GuixSD bug, indeed.

> After reboot I tried another attempt to make a 'guix system
> reconfigure', but because of /etc/group.lock and /etc/passwd.lock
> operating-system groups and users fields didn't applied.
>
> I tried to get a creation of adbusers group and add my user to it, but
> because of those files I cannot do it, until manually removed them.
>
> So, do we need to check if *.lock files are exist and remove them at the
> start of 'guix system reconfigure' OR abort reconfigure and creation of
> Grub boot entry OR something else?

I’m not sure I understand what happened.  Could you copy/paste commands
and outputs, or transcribe what happened?

TIA,
Ludo’.

Message #14 received at 28772 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Oleg Pykhalov <go.wigust <at> gmail.com>
Cc: 28772 <at> debbugs.gnu.org
Subject: Re: bug#28772: guix system reconfigure after kernel panic user or
 group not created
Date: Wed, 11 Oct 2017 15:27:18 +0200

Oleg Pykhalov <go.wigust <at> gmail.com> skribis:

> Oleg Pykhalov <go.wigust <at> gmail.com> writes:
>
> [...]
>
>> Getting it quite often while 'guix system reconfigure' at Linux magnolia
>> 4.13.4-gnu #1 SMP 1 x86_64 GNU/Linux.  But it's not a subject of current
>> report.
>
> Kernel panic was caused by my miss configuring.  I used a Guile module
> on reconfigure via GUILE_LOAD_PATH and (use-modles (my-module) in
> config.scm.  I guess it's also required after reconfigure, according to
> some times getting errors after reconfigure: cannot find my-module.  So
> I need to add my-module globally on system.

Without seeing the config, I cannot tell whether it’s a bug on our side
or not.

However, if you’re confident that the bug is in your code, that’s good.
;-)  Should we close this bug?

Thanks,
Ludo’.

Message #17 received at 28772 <at> debbugs.gnu.org (full text, mbox):

From: Oleg Pykhalov <go.wigust <at> gmail.com>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 28772 <at> debbugs.gnu.org
Subject: Re: bug#28772: guix system reconfigure after kernel panic user or
 group not created
Date: Wed, 11 Oct 2017 20:19:36 +0300

[Message part 1 (text/plain, inline)]

Hello Ludovic,

apologies for not adding logs before.  It's hard to do when I do guix
commands from Xterm and not from Emacs.  Emacs *shell* or *compilation*
buffers will eat all memory if they get too much text.

Probably need to redirect STDOUT STDERR in file when Xterm do guix.

I heard Guix folks work on implementing tiny log output to console and
redirect everything else to a log file.  This will be my life saver.

ludo <at> gnu.org (Ludovic Courtès) writes:

> Hello,
>
> Oleg Pykhalov <go.wigust <at> gmail.com> skribis:
>
>> During 'guix system reconfigure' I got a kernel panic.
>
> Can you show the exact command and its output?

Sorry, as I said this is not a topic and I don't want to do it again and
I caution to make it on my current system.

I will setup a specific Guix VM for this, where I could make a 'system
reconfigure'.  Then I'll create a new bug report with full log.

Neverless I'll leave a how-to reproduce it below for at least for myself
TODO list.

The problem
===========

The bigger problem from my view are files like /etc/group.lock and
/etc/passwd.lock.  For example:

    sudo touch /etc/group.lock

/etc/config.scm

    (operating-system
      ;; …
      (groups (cons
         (user-group (name "test"))
         %base-groups)))

reconfigure log

    

[report.send (text/plain, attachment)]

[Message part 3 (text/plain, inline)]

The new system generation was produced without "test" group, so you
could reboot into it.  And it could lead to problems if we will have a
tiny output to console and big output to a log file, I guess.

> A user-land program is not supposed to be able to cause a kernel panic;
> if it does, that’s a kernel bug.

                       How to make a kernel panic

The problem will be No defined variable IPTABLES-SSH after 'guix system
reconfigure' and kernel crash after.

$HOME/src/iptables/iptables/ru.scm

    (define-module (iptables ru)
      ;; …
      )

    (define %iptables-ssh
       "-A INPUT -p tcp --dport 22 \
    -m state --state NEW -m recent --set --name SSH -j ACCEPT")

/etc/config.scm

    (use-modules ;; …
                 (iptables ru))

    (define start-firewall
      #~(let ((iptables
               (lambda (str)
                 (zero? (system (string-join `(,#$(file-append iptables
                                                               "/sbin/iptables")
                                               ,str) " "))))))
          (format #t "Install iptables rules.~%")
          (and
           ;; …
           (iptables %iptables-ssh))))

    (define firewall-service
      (simple-service 'firewall shepherd-root-service-type
                      (list
                       (shepherd-service
                        (provision '(firewall))
                        (requirement '())
                        (start #~(lambda _
                                   #$start-firewall))
                        (respawn? #f)
                        (stop #~(lambda _
                                  (zero?
                                   (system* #$(file-append iptables
                                                           "/sbin/iptables")
                                            "-F"))))))))

    (operating-system
      ;; …
        (services (cons* ;; …
                         firewall-service)))


Make a kernel panic

    sudo GUILE_LOAD_PATH=\"$HOME/src/iptables\
    :$GUILE_LOAD_PATH\" guix system reconfigure \
    $HOME/dotfiles/guix/system-magnolia.scm

    # Run above again and kernel will panic.

> But perhaps you got the kernel panic *after* rebooting in the
> reconfigured system?  That could well be a GuixSD bug, indeed.

No, it happens after second 'guix system reconfigure' with howto above.

[...]

Thanks,
Oleg.

Message #20 received at 28772 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Oleg Pykhalov <go.wigust <at> gmail.com>
Cc: 28772 <at> debbugs.gnu.org
Subject: Re: bug#28772: guix system reconfigure after kernel panic user or
 group not created
Date: Thu, 12 Oct 2017 09:57:09 +0200

[Message part 1 (text/plain, inline)]

Hi Oleg,

Oleg Pykhalov <go.wigust <at> gmail.com> skribis:

> apologies for not adding logs before.  It's hard to do when I do guix
> commands from Xterm and not from Emacs.  Emacs *shell* or *compilation*
> buffers will eat all memory if they get too much text.

I sympathize…

> The problem
> ===========
>
> The bigger problem from my view are files like /etc/group.lock and
> /etc/passwd.lock.  For example:
>
>     sudo touch /etc/group.lock
>
> /etc/config.scm
>
>     (operating-system
>       ;; …
>       (groups (cons
>          (user-group (name "test"))
>          %base-groups)))
>
> reconfigure log

I think we can avoid the problem by forcefully removing these two lock
files at boot time:

[Message part 2 (text/x-patch, inline)]

diff --git a/gnu/services.scm b/gnu/services.scm
index 329b7b151..2ef1d8530 100644
--- a/gnu/services.scm
+++ b/gnu/services.scm
@@ -368,6 +368,8 @@ boot."
                                                 #t))))
                     ;; Ignore I/O errors so the system can boot.
                     (fail-safe
+                     (delete-file "/etc/group.lock")
+                     (delete-file "/etc/passwd.lock")
                      (delete-file-recursively "/tmp")
                      (delete-file-recursively "/var/run")
                      (mkdir "/tmp")

[Message part 3 (text/plain, inline)]

>                        How to make a kernel panic
>
> The problem will be No defined variable IPTABLES-SSH after 'guix system
> reconfigure' and kernel crash after.

[...]

> Make a kernel panic
>
>     sudo GUILE_LOAD_PATH=\"$HOME/src/iptables\
>     :$GUILE_LOAD_PATH\" guix system reconfigure \
>     $HOME/dotfiles/guix/system-magnolia.scm
>
>     # Run above again and kernel will panic.

I tried to reproduce it with my user’s shepherd, but that didn’t work:

--8<---------------cut here---------------start------------->8---
ludo <at> ribbon ~/src/guix$ cat ,t.scm
(define s
  (make <service>
    #:provides '(nothing)
    #:start (lambda _ unbound)))

(register-services s)
(start s)
ludo <at> ribbon ~/src/guix$ herd load root ,t.scm
Loading ,t.scm.
herd: exception caught while executing 'load' on service 'root':
ERROR: Unbound variable: foo
ludo <at> ribbon ~/src/guix$ herd load root ,t.scm
Loading ,t.scm.
herd: exception caught while executing 'start' on service 'nothing':
ERROR: Unbound variable: unbound
ludo <at> ribbon ~/src/guix$ herd load root ,t.scm
Loading ,t.scm.
Assertion (null? (lookup-services (canonical-name new))) failed.
herd: exception caught while executing 'load' on service 'root':
ERROR: Throw to key `assertion-failed' with args `()'.
ludo <at> ribbon ~/src/guix$ echo $?
1
ludo <at> ribbon ~/src/guix$ herd status

[...]

ludo <at> ribbon ~/src/guix$ echo $?
0
--8<---------------cut here---------------end--------------->8---

IOW, shepherd caught the exceptions and didn’t die.

What am I missing?

Ludo’.

Message #23 received at 28772 <at> debbugs.gnu.org (full text, mbox):

From: Oleg Pykhalov <go.wigust <at> gmail.com>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 28772 <at> debbugs.gnu.org
Subject: Re: bug#28772: guix system reconfigure after kernel panic user or
 group not created
Date: Thu, 12 Oct 2017 11:15:48 +0300

Hello Ludovic,

ludo <at> gnu.org (Ludovic Courtès) writes:

[...]

> I think we can avoid the problem by forcefully removing these two lock
> files at boot time:
>
> diff --git a/gnu/services.scm b/gnu/services.scm
> index 329b7b151..2ef1d8530 100644
> --- a/gnu/services.scm
> +++ b/gnu/services.scm
> @@ -368,6 +368,8 @@ boot."
>                                                  #t))))
>                      ;; Ignore I/O errors so the system can boot.
>                      (fail-safe
> +                     (delete-file "/etc/group.lock")
> +                     (delete-file "/etc/passwd.lock")
>                       (delete-file-recursively "/tmp")
>                       (delete-file-recursively "/var/run")
>                       (mkdir "/tmp")
>
>

There is also a '/etc/.pwd.lock'.  Info about this file
https://lists.debian.org/debian-user/2005/07/msg02949.html


I'm not sure if any files are exist.  Days past after reconfigure
failure.

$ sudo find /etc -name '*.lock' # Shows nothing.

[...]

> IOW, shepherd caught the exceptions and didn’t die.
>
> What am I missing?

I'll try to make a reproducible thing later.

Thanks,
Oleg.

Message #26 received at 28772 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Oleg Pykhalov <go.wigust <at> gmail.com>
Cc: 28772 <at> debbugs.gnu.org
Subject: Re: bug#28772: guix system reconfigure after kernel panic user or
 group not created
Date: Fri, 13 Oct 2017 10:25:52 +0200

Hi Oleg,

Oleg Pykhalov <go.wigust <at> gmail.com> skribis:

> ludo <at> gnu.org (Ludovic Courtès) writes:
>
> [...]
>
>> I think we can avoid the problem by forcefully removing these two lock
>> files at boot time:
>>
>> diff --git a/gnu/services.scm b/gnu/services.scm
>> index 329b7b151..2ef1d8530 100644
>> --- a/gnu/services.scm
>> +++ b/gnu/services.scm
>> @@ -368,6 +368,8 @@ boot."
>>                                                  #t))))
>>                      ;; Ignore I/O errors so the system can boot.
>>                      (fail-safe
>> +                     (delete-file "/etc/group.lock")
>> +                     (delete-file "/etc/passwd.lock")
>>                       (delete-file-recursively "/tmp")
>>                       (delete-file-recursively "/var/run")
>>                       (mkdir "/tmp")
>>
>>
>
> There is also a '/etc/.pwd.lock'.  Info about this file
> https://lists.debian.org/debian-user/2005/07/msg02949.html
>
>
> I'm not sure if any files are exist.  Days past after reconfigure
> failure.
>
> $ sudo find /etc -name '*.lock' # Shows nothing.

I’ve pushed it as aad8a143000600abec5c8ebfadec4c09f34f1b73.

> [...]
>
>> IOW, shepherd caught the exceptions and didn’t die.
>>
>> What am I missing?
>
> I'll try to make a reproducible thing later.

Awesome.

Thanks,
Ludo’.

Message #29 received at 28772 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: guix-devel <at> gnu.org
Cc: 28772 <at> debbugs.gnu.org, Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Subject: Re: 02/02: doc: Add an example to the documentation of the
 udev-service.
Date: Fri, 13 Oct 2017 10:35:43 +0200

Hi!

rekado <at> elephly.net (Ricardo Wurmus) skribis:

> commit e0c1d080b520c1bbd2dcd7bc90a750f5ce580486
> Author: Ricardo Wurmus <rekado <at> elephly.net>
> Date:   Mon Oct 9 23:03:56 2017 +0200
>
>     doc: Add an example to the documentation of the udev-service.
>     
>     * doc/guix.texi (Base Services): Update 'udev-service' documentation.

Good idea.

> +@example
> +(define %example-udev-rule
> +  (udev-rule "90-usb-thing.rules"
> +             "ACTION==\"add\", SUBSYSTEM==\"usb\", ATTR@{product@}==\"Example\", RUN+=\"/path/to/script\""))
> +
> +(operating-system
> +  ;; @dots{}
> +  (services (modify-services %desktop-services
> +              (udev-service-type config =>
> +                (udev-configuration (inherit config)
> +                  (rules (append (udev-configuration-rules config)
> +                                 (list %example-udev-rule))))))))
> +@end example

<https://bugs.gnu.org/28647> is somewhat related.

Fundamentally though, to simplify this use case, we should have:

  (define (additional-udev-rules . rules)
    "Add RULES, a list of file-like object, as a udev rules."
    (simple-service 'udev-rule udev-service-type rules))

so one can write:

  (operating-system
     ;; …
     (services (cons (additional-udev-rules %example-udev-rule)
                     %desktop-services)))

Thoughts?

Ludo’.

Message #32 received at 28772 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Oleg Pykhalov <go.wigust <at> gmail.com>
Cc: 28772 <at> debbugs.gnu.org
Subject: Re: bug#28772: guix system reconfigure after kernel panic user or
 group not created
Date: Fri, 20 Oct 2017 18:02:08 +0200

Hi,

Oleg Pykhalov <go.wigust <at> gmail.com> skribis:

> ludo <at> gnu.org (Ludovic Courtès) writes:
>
> [...]
>
>> I think we can avoid the problem by forcefully removing these two lock
>> files at boot time:
>>
>> diff --git a/gnu/services.scm b/gnu/services.scm
>> index 329b7b151..2ef1d8530 100644
>> --- a/gnu/services.scm
>> +++ b/gnu/services.scm
>> @@ -368,6 +368,8 @@ boot."
>>                                                  #t))))
>>                      ;; Ignore I/O errors so the system can boot.
>>                      (fail-safe
>> +                     (delete-file "/etc/group.lock")
>> +                     (delete-file "/etc/passwd.lock")
>>                       (delete-file-recursively "/tmp")
>>                       (delete-file-recursively "/var/run")
>>                       (mkdir "/tmp")
>>
>>
>
> There is also a '/etc/.pwd.lock'.  Info about this file
> https://lists.debian.org/debian-user/2005/07/msg02949.html
>
>
> I'm not sure if any files are exist.  Days past after reconfigure
> failure.
>
> $ sudo find /etc -name '*.lock' # Shows nothing.
>
> [...]
>
>> IOW, shepherd caught the exceptions and didn’t die.
>>
>> What am I missing?
>
> I'll try to make a reproducible thing later.

Did you eventually gather more info?

Thanks,
Ludo’.

Message #37 received at 28772 <at> debbugs.gnu.org (full text, mbox):

From: Oleg Pykhalov <go.wigust <at> gmail.com>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 28772 <at> debbugs.gnu.org
Subject: Re: bug#28772: guix system reconfigure after kernel panic user or
 group not created
Date: Sun, 22 Oct 2017 17:41:34 +0300

[Message part 1 (text/plain, inline)]

Hello Ludovic,

Apologies for the late reply.

ludo <at> gnu.org (Ludovic Courtès) writes:

[...]

> Did you eventually gather more info?

Yes, I got an undefined %iptables-rst variable in reconfigure output
at first run.

But at second run substitutions didn't work.  GuixSD rebuilds the world.
I didn't wait for this.

I also didn't make guix pull.

/tmp/guixsd
├── bootstrap.sh
├── iptables
│   └── iptables.scm
├── panic.sh
└── vm-image.scm

1 directory, 4 files

[bootstrap.sh (application/x-sh, attachment)]

[panic.sh (application/x-sh, attachment)]

[vm-image.scm (application/octet-stream, attachment)]

[iptables.scm (application/octet-stream, attachment)]

Message #40 received at 28772 <at> debbugs.gnu.org (full text, mbox):

From: Oleg Pykhalov <go.wigust <at> gmail.com>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 28772 <at> debbugs.gnu.org
Subject: Re: bug#28772: guix system reconfigure after kernel panic user or
 group not created
Date: Sun, 22 Oct 2017 23:39:55 +0300

ludo <at> gnu.org (Ludovic Courtès) writes:

[...]

> Did you eventually gather more info?

In addition to previous message.

--- /home/natsu/r/bootstrap.sh	2017-10-22 04:48:42.992394510 +0300
+++ ./bootstrap.sh	2017-10-22 23:33:54.113495658 +0300
@@ -11,7 +11,7 @@
 SIG=guixsd-vm-image-$VERSION.$KERNEL.xz.sig
 
 pull_release () {
-    if [ ! -f $RELEASE ] || [ ! -f $RELEASE.xz ] || ! gpg --verify $SIG
+    if [ ! -f $RELEASE ] || [ ! -f $RELEASE.xz ]
     then
         gpg --keyserver pgp.mit.edu --recv-keys $GPG_KEY
         wget --output-document=$RELEASE.xz https://alpha.gnu.org/gnu/guix/$RELEASE.xz 
@@ -23,6 +23,7 @@
 
 if pull_release
 then qemu-system-x86_64 \
+     -enable-kvm \
      -daemonize \
      -m 1024 \
      -virtfs local,path=$PWD,security_model=none,mount_tag=TAG_pwd \

Also need to increase a size of qemu image.  I don't know how I
reconfigured successfully in previous message.

Message #43 received at 28772 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Oleg Pykhalov <go.wigust <at> gmail.com>
Cc: 28772 <at> debbugs.gnu.org
Subject: Re: bug#28772: guix system reconfigure after kernel panic user or
 group not created
Date: Sun, 22 Oct 2017 16:01:39 -0700

Hi Oleg,

Oleg Pykhalov <go.wigust <at> gmail.com> skribis:

> ludo <at> gnu.org (Ludovic Courtès) writes:
>
> [...]
>
>> Did you eventually gather more info?
>
> Yes, I got an undefined %iptables-rst variable in reconfigure output
> at first run.
>
> But at second run substitutions didn't work.  GuixSD rebuilds the world.
> I didn't wait for this.

In my previous message I showed a way to (attempt to) reproduce the
problem by directly using ‘herd load’.

Could you try to reproduce the problem in this way?

You should even be able to check with a shepherd instance not running as
PID 1, which is more convenient.

Thanks in advance,
Ludo’.

Message #46 received at 28772 <at> debbugs.gnu.org (full text, mbox):

From: zimoun <zimon.toutoune <at> gmail.com>
To: Oleg Pykhalov <go.wigust <at> gmail.com>
Cc: 28772 <at> debbugs.gnu.org
Subject: Re: bug#28772: guix system reconfigure after kernel panic user or
 group not created
Date: Fri, 26 Nov 2021 02:19:09 +0100

Hi,

On Tue, 10 Oct 2017 at 08:51, Oleg Pykhalov <go.wigust <at> gmail.com> wrote:

> During 'guix system reconfigure' I got a kernel panic.
>
> Getting it quite often while 'guix system reconfigure' at Linux magnolia
> 4.13.4-gnu #1 SMP 1 x86_64 GNU/Linux.  But it's not a subject of current
> report.
>
> After reboot I tried another attempt to make a 'guix system
> reconfigure', but because of /etc/group.lock and /etc/passwd.lock
> operating-system groups and users fields didn't applied.
>
> I tried to get a creation of adbusers group and add my user to it, but
> because of those files I cannot do it, until manually removed them.
>
> So, do we need to check if *.lock files are exist and remove them at the
> start of 'guix system reconfigure' OR abort reconfigure and creation of
> Grub boot entry OR something else?


The last message for bug 28772 [1] is 22 Oct 2017 and it is marked
’moreinfo’ since 2017.  I propose to close or provide more details.

1: <http://issues.guix.gnu.org/issue/28772>


Cheers,
simon

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.