GNU bug report logs - #19190
Cannot boot with encrypted root

Previous Next

Package: guix;

Reported by: Nikita Karetnikov <nikita <at> karetnikov.org>

Date: Wed, 26 Nov 2014 04:28:01 UTC

Severity: normal

Done: ludo <at> gnu.org (Ludovic Courtès)

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 19190 in the body.
You can then email your comments to 19190 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-guix <at> gnu.org:
bug#19190; Package guix. (Wed, 26 Nov 2014 04:28:01 GMT) Full text and rfc822 format available.

Acknowledgement sent to Nikita Karetnikov <nikita <at> karetnikov.org>:
New bug report received and forwarded. Copy sent to bug-guix <at> gnu.org. (Wed, 26 Nov 2014 04:28:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Nikita Karetnikov <nikita <at> karetnikov.org>
To: bug-guix <at> gnu.org
Subject: Cannot boot with encrypted root
Date: Wed, 26 Nov 2014 07:26:50 +0400
[Message part 1 (text/plain, inline)]
(Ludo suggested to report this as a bug, so this issue won’t get lost.)

While booting, GRUB prints this error:

error: file
‘/gnu/store/rz…-linux-libre-3.17.3/bzImage’ not found.

which is not surprising since the store is encrypted.

I’ve tried putting these lines after the “setparams” line in the GRUB
config (by hitting the ‘e’ key):

insmod luks
cryptomount hd0,gpt3
set root=crypto0

After pressing the ‘F10’ key, I get a password prompt.  And after that, a
bunch of warnings about ‘/dev/mapper/main’:

ext2fs_check_if_mount: Can’t check if filesystem is mounted due to
missing mtab file while determining whether /dev/mapper/main is mounted.
fsck.ext3: No such file or directory while trying to open
/dev/mapper/main
Possibly non-existent device?
‘fsck.ext3’ exited with code 8 on /dev/mapper/main; spawning REPL

In the REPL:

scheme@(guile-user)> ,use (gnu build linux-boot)
scheme@(guile-user)> (boot-system)
[…]
ERROR: In procedure mount:
ERROR: In procedure mount: Device or resource busy

My config is shown below:

(use-modules (gnu))

(define %linux-modules
  '(
    ;; cryptsetup/LUKS
    "dm-crypt.ko" "aes-x86_64.ko" "crc32.ko" "pcbc.ko" "xcbc.ko" "sha256-ssse3.ko"))

(operating-system
  (host-name "test")
  (timezone "Europe/Paris")
  (locale "en_US.UTF-8")

  (bootloader (grub-configuration
               (device "/dev/sda")))

  (initrd (lambda (fs . args)
            (apply base-initrd fs
                   #:extra-modules %linux-modules
                   args)))

  (mapped-devices (list (mapped-device
                         (source "/dev/sda3")
                         (target "main")
                         (type luks-device-mapping))))

  (file-systems (cons* (file-system
                         (device "/dev/mapper/main")
                         (mount-point "/")
                         (type "ext3"))
                       (file-system
                         (device "boot")
                         (type 'label)
                         (mount-point "/boot")
                         (type "ext3"))
                       %base-file-systems))

  (users (list (user-account
                (name "test")
                (group "users")

                (supplementary-groups '("wheel"
                                        "audio" "video"))
                (home-directory "/home/test")))))
[Message part 2 (application/pgp-signature, inline)]

Information forwarded to bug-guix <at> gnu.org:
bug#19190; Package guix. (Wed, 26 Nov 2014 09:56:02 GMT) Full text and rfc822 format available.

Message #8 received at 19190 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Nikita Karetnikov <nikita <at> karetnikov.org>
Cc: 19190 <at> debbugs.gnu.org
Subject: Re: bug#19190: Cannot boot with encrypted root
Date: Wed, 26 Nov 2014 10:55:03 +0100
Thanks for the report.  Let’s address the initrd/boot issue first.

Nikita Karetnikov <nikita <at> karetnikov.org> skribis:

> After pressing the ‘F10’ key, I get a password prompt.

Any message after entering the password prompt?

> And after that, a bunch of warnings about ‘/dev/mapper/main’:
>
> ext2fs_check_if_mount: Can’t check if filesystem is mounted due to
> missing mtab file while determining whether /dev/mapper/main is mounted.
> fsck.ext3: No such file or directory while trying to open
> /dev/mapper/main
> Possibly non-existent device?
> ‘fsck.ext3’ exited with code 8 on /dev/mapper/main; spawning REPL

It looks like ‘cryptsetup open’ simply failed, and ‘boot-system’
currently ignores that error, so it goes up to the file system check as
shown above, which in turn fails and gets us to the debugger.

Perhaps you could try running ‘cryptsetup’ by hand from the REPL.
That’s going to be a bit tedious; something like:

  (use-modules (srfi srfi-1) (ice-9 ftw))
  (define cs (find (lambda (file)
                     (string-contains file "cryptsetup"))
                   (scandir "/gnu/store")))
  (system* (string-append "/gnu/store/" cs "/bin/cryptsetup")
           "open" "--type" "luks" "/dev/whatever" "main")

Then, please report the return value of ‘system*’, and check

  (file-exists? "/dev/mapper/main")

Please also report any messages.

Thanks in advance!

Ludo’.




Information forwarded to bug-guix <at> gnu.org:
bug#19190; Package guix. (Wed, 26 Nov 2014 12:42:01 GMT) Full text and rfc822 format available.

Message #11 received at 19190 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Nikita Karetnikov <nikita <at> karetnikov.org>
Cc: 19190 <at> debbugs.gnu.org
Subject: Re: bug#19190: Cannot boot with encrypted root
Date: Wed, 26 Nov 2014 13:41:27 +0100
I noticed an obvious issue in (gnu build linux-boot), which is that
device-mapping (the #:pre-mount argument) would be performed after
‘mount-root-file-system’ has been called.

Commit 67979e4 fixes that.

To fix it, we’ll have to update ‘guix-devel’ in (gnu packages
package-management), and to regenerate a VM image that uses it.  I can
help with that.

Ludo’.




Information forwarded to bug-guix <at> gnu.org:
bug#19190; Package guix. (Wed, 26 Nov 2014 16:05:02 GMT) Full text and rfc822 format available.

Message #14 received at 19190 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Duncan Keall <duncan <at> duncankeall.com>
Cc: Nikita Karetnikov <nikita <at> karetnikov.org>, 19190 <at> debbugs.gnu.org
Subject: Re: bug#19190: Cannot boot with encrypted root
Date: Wed, 26 Nov 2014 17:04:51 +0100
Hi,  :-)

Duncan Keall <duncan <at> duncankeall.com> skribis:

>> Perhaps you could try running ‘cryptsetup’ by hand from the REPL.
>
> I needed to replace "/bin/cryptsetup" with "/sbin/cryptsetup". After
> that, cryptsetup accepted a password and printed these errors:
>
>   device-mapper: table 253:0: crypt: Error allocating crypto tfm
>   device-mapper: ioctl: error adding target to table
>   device-mapper: reload ioctl on  failed: No such file or directory
>   Failed to open temporary keystore device.
>   device-mapper: remove ioctl on temporary-cryptsetup-149 failed: No
>   such device or address

I suspect this is due to missing modules.

Does your initrd configuration have the modules Nikita’s configuration
shows?

Thanks for testing so quickly!

Ludo’.




Information forwarded to bug-guix <at> gnu.org:
bug#19190; Package guix. (Wed, 26 Nov 2014 16:41:03 GMT) Full text and rfc822 format available.

Message #17 received at 19190 <at> debbugs.gnu.org (full text, mbox):

From: Duncan Keall <duncan <at> duncankeall.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Nikita Karetnikov <nikita <at> karetnikov.org>, 19190 <at> debbugs.gnu.org
Subject: Re: bug#19190: Cannot boot with encrypted root
Date: Thu, 27 Nov 2014 01:05:17 +1300
Here's what I've found using a very similar setup to Nikita and after
receiving the same error messages.

> Any message after entering the password prompt?

GRUB prints a single message "Slot 0 opened" and then it's into the
kernel boot logs. The first related message appears to be the
"ext2fs_check_if_mount..." one.

> Perhaps you could try running ‘cryptsetup’ by hand from the REPL.

I needed to replace "/bin/cryptsetup" with "/sbin/cryptsetup". After
that, cryptsetup accepted a password and printed these errors:

  device-mapper: table 253:0: crypt: Error allocating crypto tfm
  device-mapper: ioctl: error adding target to table
  device-mapper: reload ioctl on  failed: No such file or directory
  Failed to open temporary keystore device.
  device-mapper: remove ioctl on temporary-cryptsetup-149 failed: No
  such device or address

The return value was 256, and afterwards /dev/mapper/main still did not
exist.




Information forwarded to bug-guix <at> gnu.org:
bug#19190; Package guix. (Thu, 27 Nov 2014 02:07:02 GMT) Full text and rfc822 format available.

Message #20 received at 19190 <at> debbugs.gnu.org (full text, mbox):

From: Duncan Keall <duncan <at> duncankeall.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Nikita Karetnikov <nikita <at> karetnikov.org>, 19190 <at> debbugs.gnu.org
Subject: Re: bug#19190: Cannot boot with encrypted root
Date: Thu, 27 Nov 2014 15:06:03 +1300
> Does your initrd configuration have the modules Nikita’s configuration
> shows?

Here are the configurations I have tested which all result in the errors
posted above:

  ("dm-crypt.ko" "aes-x86_64.ko" "crc32.ko" "pcbc.ko" "xcbc.ko" "sha256-ssse3.ko")
  ("dm-crypt.ko" "cryptd.ko" "aes-x86_64.ko" "crc32.ko" "pcbc.ko" "xcbc.ko" "sha256-ssse3.ko")
  ("dm-crypt.ko" "cryptd.ko" "crc32-pclmul.ko" "aes-x86_64.ko" "crc32.ko" "pcbc.ko" "xcbc.ko" "sha256-ssse3.ko")

I could also add "aesni-intel.ko" but it resulted in a different set of
kernel errors on boot:

  aesni_intel: Unknown symbol ablk_decrypt (err 0)
  aesni_intel: Unknown symbol lrw_free_table (err 0)
  aesni_intel: Unknown symbol ablk_set_key (err 0)
  ...
  ERROR: In procedure load-linux-module: Unknown error -1

For reference, here are all the modules loaded on my current system
which might be related:

  aes_x86_64 aesni_intel af_alg algif_skcipher arc4 ccm crc32_pclmul
  crc32c_intel crc_t10dif crct10dif_common crct10dif_pclmul cryptd ctr
  dm_crypt dm_mod gf128mul ghash_clmulni_intel lrw xor




Information forwarded to bug-guix <at> gnu.org:
bug#19190; Package guix. (Thu, 27 Nov 2014 10:13:02 GMT) Full text and rfc822 format available.

Message #23 received at 19190 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Duncan Keall <duncan <at> duncankeall.com>
Cc: Nikita Karetnikov <nikita <at> karetnikov.org>, 19190 <at> debbugs.gnu.org
Subject: Re: bug#19190: Cannot boot with encrypted root
Date: Thu, 27 Nov 2014 11:12:39 +0100
Duncan Keall <duncan <at> duncankeall.com> skribis:

> Here are the configurations I have tested which all result in the errors
> posted above:
>
>   ("dm-crypt.ko" "aes-x86_64.ko" "crc32.ko" "pcbc.ko" "xcbc.ko" "sha256-ssse3.ko")
>   ("dm-crypt.ko" "cryptd.ko" "aes-x86_64.ko" "crc32.ko" "pcbc.ko" "xcbc.ko" "sha256-ssse3.ko")
>   ("dm-crypt.ko" "cryptd.ko" "crc32-pclmul.ko" "aes-x86_64.ko" "crc32.ko" "pcbc.ko" "xcbc.ko" "sha256-ssse3.ko")
>
> I could also add "aesni-intel.ko" but it resulted in a different set of
> kernel errors on boot:
>
>   aesni_intel: Unknown symbol ablk_decrypt (err 0)
>   aesni_intel: Unknown symbol lrw_free_table (err 0)
>   aesni_intel: Unknown symbol ablk_set_key (err 0)
>   ...
>   ERROR: In procedure load-linux-module: Unknown error -1

Presumably that’s because one of its prerequisites hasn’t been loaded
yet.

The difficulty is that modules need to be listed in topological order:
things that aesni_intel.ko depends on must appear before aesni_intel.ko.

On my machine, I see:

--8<---------------cut here---------------start------------->8---
$ lsmod|grep aes
aesni_intel           165373  3 
ablk_helper            13597  1 aesni_intel
cryptd                 20359  4 ghash_clmulni_intel,aesni_intel,ablk_helper
lrw                    13286  1 aesni_intel
glue_helper            13990  1 aesni_intel
aes_x86_64             17131  1 aesni_intel
--8<---------------cut here---------------end--------------->8---

So the right order for these would be:

  cryptd
  aes_x86_64
  ablk_helper
  lrw
  aesni_intel

We’ll have to change the initrd build to figure that out automatically,
but in the meantime, we have to sort them by hand.

Ludo’.




Information forwarded to bug-guix <at> gnu.org:
bug#19190; Package guix. (Sat, 13 Dec 2014 22:02:01 GMT) Full text and rfc822 format available.

Message #26 received at 19190 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Duncan Keall <duncan <at> duncankeall.com>
Cc: Nikita Karetnikov <nikita <at> karetnikov.org>, 19190 <at> debbugs.gnu.org
Subject: Re: bug#19190: Cannot boot with encrypted root
Date: Sat, 13 Dec 2014 23:01:16 +0100
ludo <at> gnu.org (Ludovic Courtès) skribis:

> Duncan Keall <duncan <at> duncankeall.com> skribis:
>
>> Here are the configurations I have tested which all result in the errors
>> posted above:
>>
>>   ("dm-crypt.ko" "aes-x86_64.ko" "crc32.ko" "pcbc.ko" "xcbc.ko" "sha256-ssse3.ko")
>>   ("dm-crypt.ko" "cryptd.ko" "aes-x86_64.ko" "crc32.ko" "pcbc.ko" "xcbc.ko" "sha256-ssse3.ko")
>>   ("dm-crypt.ko" "cryptd.ko" "crc32-pclmul.ko" "aes-x86_64.ko" "crc32.ko" "pcbc.ko" "xcbc.ko" "sha256-ssse3.ko")
>>
>> I could also add "aesni-intel.ko" but it resulted in a different set of
>> kernel errors on boot:
>>
>>   aesni_intel: Unknown symbol ablk_decrypt (err 0)
>>   aesni_intel: Unknown symbol lrw_free_table (err 0)
>>   aesni_intel: Unknown symbol ablk_set_key (err 0)
>>   ...
>>   ERROR: In procedure load-linux-module: Unknown error -1
>
> Presumably that’s because one of its prerequisites hasn’t been loaded
> yet.
>
> The difficulty is that modules need to be listed in topological order:

This particular problem was fixed in 0e704a2 a couple of weeks ago.

For testing, you would need to start from an image generated from
current master with:

  ./pre-inst-env \
    guix system disk-image --image-size=800MiB gnu/system/install.scm

Let me know if anything else is needed.

Ludo’.




Information forwarded to bug-guix <at> gnu.org:
bug#19190; Package guix. (Mon, 15 Dec 2014 10:50:02 GMT) Full text and rfc822 format available.

Message #29 received at 19190 <at> debbugs.gnu.org (full text, mbox):

From: Duncan Keall <duncan <at> duncankeall.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Nikita Karetnikov <nikita <at> karetnikov.org>, 19190 <at> debbugs.gnu.org
Subject: Re: bug#19190: Cannot boot with encrypted root
Date: Mon, 15 Dec 2014 23:49:39 +1300
Thanks for the update, Ludovic!

I've just tested another install with a fresh image from master, using
as similar setup as possible to before.

The boot process still fails at mounting the root filesystem (as
expected):

  fsck.ext4: No such file or directory while trying to open /dev/mapper/main
  Possibly non-existent device?

However I found that cryptsetup was missing from the store during the
early-boot REPL, so I wasn't able to test manually mounting the
encrypted filesystem. 

Nothing in the commit history since v0.8 jumped out as being
responsible, so I'm assuming I've missed something obvious during
install. I'll have to keep looking!




Reply sent to ludo <at> gnu.org (Ludovic Courtès):
You have taken responsibility. (Sun, 20 Sep 2015 20:15:01 GMT) Full text and rfc822 format available.

Notification sent to Nikita Karetnikov <nikita <at> karetnikov.org>:
bug acknowledged by developer. (Sun, 20 Sep 2015 20:15:02 GMT) Full text and rfc822 format available.

Message #34 received at 19190-done <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Duncan Keall <duncan <at> duncankeall.com>
Cc: Nikita Karetnikov <nikita <at> karetnikov.org>, 19190-done <at> debbugs.gnu.org,
 Petter <petter <at> mykolab.ch>
Subject: Re: bug#19190: Cannot boot with encrypted root
Date: Sun, 20 Sep 2015 22:14:33 +0200
Duncan Keall <duncan <at> duncankeall.com> skribis:

> The boot process still fails at mounting the root filesystem (as
> expected):
>
>   fsck.ext4: No such file or directory while trying to open /dev/mapper/main
>   Possibly non-existent device?

It took us a while but this part is now fixed with cc0e575, and commit
07779d0 adds dm-crypt.ko and xts.ko and their dependencies to the
default initrd.

I’m closing this bug, but if you have time and interest, you’re welcome
to confirm whether this works for you.

Thanks!

Ludo’.




bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Mon, 19 Oct 2015 11:24:04 GMT) Full text and rfc822 format available.

This bug report was last modified 8 years and 163 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.