GNU bug report logs -
#34902
Kernel module not found at boot time due to hyphen/underscore mismatch
Previous Next
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 34902 in the body.
You can then email your comments to 34902 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
bug-guix <at> gnu.org
:
bug#34902
; Package
guix
.
(Mon, 18 Mar 2019 09:32:02 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
Julien Lepiller <julien <at> lepiller.eu>
:
New bug report received and forwarded. Copy sent to
bug-guix <at> gnu.org
.
(Mon, 18 Mar 2019 09:32:02 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
Hi!
I've installed the Guix system on my cubietruck yesterday, but had some
difficulties. At first, the root partition was not available at boot, so
I got a repl with a message saying that /dev/mmcblk0p1 was not
available. I added the following to my guix config file:
(initrd-modules (cons "sunxi_mmc" %base-initrd-modules))
although the guix system command worked, the produced system still
couldn't boot, but this time the message was that guix was unable to
find sunxi_mmc.ko. Using the repl I could confirm that this module was
indeed here, but under the name sunxi-mmc.ko. Using load-linux-module*,
I was able to confirm that loading it made the filesystem available in
/dev. In the end this line:
(initrd-modules (cons "sunxi-mmc" %base-initrd-modules))
was the right line to add, and I could properly boot my cubietruck!
The bug here is that guix should either be smarter and load sunxi-mmc.ko
when it can't find sunxi_mmc.ko, or not allow me to build a system when
I specify sunxi_mmc since it doesn't exist at boot time.
Information forwarded
to
bug-guix <at> gnu.org
:
bug#34902
; Package
guix
.
(Mon, 18 Mar 2019 20:43:02 GMT)
Full text and
rfc822 format available.
Message #8 received at 34902 <at> debbugs.gnu.org (full text, mbox):
Hi Julien,
Julien Lepiller <julien <at> lepiller.eu> skribis:
> I've installed the Guix system on my cubietruck yesterday, but had
> some difficulties. At first, the root partition was not available at
> boot, so I got a repl with a message saying that /dev/mmcblk0p1 was
> not available. I added the following to my guix config file:
>
> (initrd-modules (cons "sunxi_mmc" %base-initrd-modules))
Didn’t ‘guix system init’ suggest adding this module? Could it be that
this module was actually built-in in the kernel you booted?
> although the guix system command worked, the produced system still
> couldn't boot, but this time the message was that guix was unable to
> find sunxi_mmc.ko. Using the repl I could confirm that this module was
> indeed here, but under the name sunxi-mmc.ko. Using
> load-linux-module*, I was able to confirm that loading it made the
> filesystem available in /dev. In the end this line:
>
> (initrd-modules (cons "sunxi-mmc" %base-initrd-modules))
>
> was the right line to add, and I could properly boot my cubietruck!
Phewww.
> The bug here is that guix should either be smarter and load
> sunxi-mmc.ko when it can't find sunxi_mmc.ko, or not allow me to build
> a system when I specify sunxi_mmc since it doesn't exist at boot time.
This underscore vs. hyphen thing is terrible.
In commit fcd068e984078ab74c6842af2525bf88096cd262 we fixed the initrd
builder so it would try both file underscore and hyphen.
But now I suppose we need to do the same in ‘load-linux-module*’?
Thanks,
Ludo’.
Information forwarded
to
bug-guix <at> gnu.org
:
bug#34902
; Package
guix
.
(Mon, 18 Mar 2019 21:35:03 GMT)
Full text and
rfc822 format available.
Message #11 received at 34902 <at> debbugs.gnu.org (full text, mbox):
Le Mon, 18 Mar 2019 21:42:29 +0100,
Ludovic Courtès <ludo <at> gnu.org> a écrit :
> Hi Julien,
>
> Julien Lepiller <julien <at> lepiller.eu> skribis:
>
> > I've installed the Guix system on my cubietruck yesterday, but had
> > some difficulties. At first, the root partition was not available at
> > boot, so I got a repl with a message saying that /dev/mmcblk0p1 was
> > not available. I added the following to my guix config file:
> >
> > (initrd-modules (cons "sunxi_mmc" %base-initrd-modules))
>
> Didn’t ‘guix system init’ suggest adding this module? Could it be
> that this module was actually built-in in the kernel you booted?
I couldn't find it in the output of lsmod, so I think it was builtin.
>
> > although the guix system command worked, the produced system still
> > couldn't boot, but this time the message was that guix was unable to
> > find sunxi_mmc.ko. Using the repl I could confirm that this module
> > was indeed here, but under the name sunxi-mmc.ko. Using
> > load-linux-module*, I was able to confirm that loading it made the
> > filesystem available in /dev. In the end this line:
> >
> > (initrd-modules (cons "sunxi-mmc" %base-initrd-modules))
> >
> > was the right line to add, and I could properly boot my
> > cubietruck!
>
> Phewww.
>
> > The bug here is that guix should either be smarter and load
> > sunxi-mmc.ko when it can't find sunxi_mmc.ko, or not allow me to
> > build a system when I specify sunxi_mmc since it doesn't exist at
> > boot time.
>
> This underscore vs. hyphen thing is terrible.
>
> In commit fcd068e984078ab74c6842af2525bf88096cd262 we fixed the initrd
> builder so it would try both file underscore and hyphen.
>
> But now I suppose we need to do the same in ‘load-linux-module*’?
I guess so :)
>
> Thanks,
> Ludo’.
Information forwarded
to
bug-guix <at> gnu.org
:
bug#34902
; Package
guix
.
(Mon, 18 Mar 2019 22:41:02 GMT)
Full text and
rfc822 format available.
Message #14 received at 34902 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
> This underscore vs. hyphen thing is terrible.
Yeah. But looking at kmod, it doesn't seem to be so bad for them.
They basically always use the version with underscores as the lookup
key for their internal hash tables.
For alias resolving, they use a resolver for [a-zxxx] AND they replace
dashes THAT ARE NOT IN THE "[...]" block by underscore before using it
for lookups.
modules.dep entries are always file names (no replacements whatsoever).
When a module is loaded as a result of an alias lookup, it has a special
key (string-append name "\" target-alias). No idea what that's for.
I'd caution about complicating our version overly much by putting
just-in-case replacements everywhere. Rather we should find out what
their original strategy is (if any :P) and do that as well.
Module aliases (the alias itself, not what it's resolved to) sometimes look
like that:
./arch/x86/crypto/des3_ede_glue.c:MODULE_ALIAS_CRYPTO("des3_ede-asm");
WTF...
But the thing the alias is resolved to is always the modname (which
has hyphens replaced by underscore and stops before a dot).
Their bin files (we don't use them) always have the final modname as key,
as opposed to their source files (for example "modules.alias" rather
than "modules.alias.bin"). (The source files sometimes have paths as key)
I think that they assume that each module, whether it has dependencies or
not, is listed in modules.dep . They use modules.dep.bin for the lookup
of a module by modname, rather than by path.
So I think a pretty good fix is:
Every time we want to lookup a module by modname, check a hashtable that
is built from modules.dep by:
* let x-path be the first value of each modules.dep line.
* Then the key for the hashtable entry is (underscore (stop-at-dot (basename x-path)))
* And the value for the hashtable entry is x-path.
In this way we would look up modules in a way similar to them.
Why they couldn't just have used file names without mangling them beats me.
Sigh...
[Message part 2 (application/pgp-signature, inline)]
Information forwarded
to
bug-guix <at> gnu.org
:
bug#34902
; Package
guix
.
(Fri, 22 Mar 2019 20:29:02 GMT)
Full text and
rfc822 format available.
Message #17 received at 34902 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Hi Danny,
Thanks for the explanation, I was unaware of all these subtleties…
Danny Milosavljevic <dannym <at> scratchpost.org> skribis:
> Every time we want to lookup a module by modname, check a hashtable that
> is built from modules.dep by:
>
> * let x-path be the first value of each modules.dep line.
> * Then the key for the hashtable entry is (underscore (stop-at-dot (basename x-path)))
> * And the value for the hashtable entry is x-path.
>
> In this way we would look up modules in a way similar to them.
Instead of guessing, we could also store a mapping from module name to
file name. I had that in a previous patch series that we discussed some
time ago (see below.)
This a custom format, not used by kmod or any other tool, but the
advantage is that it solves the module/file name mapping without ugly
guess-hacks.
WDYT?
Thanks,
Ludo’.
[Message part 2 (text/x-patch, inline)]
commit 370c4426aa94d99f8faafd3992d54169ff918fb1
Author: Ludovic Courtès <ludo <at> gnu.org>
Date: Wed Mar 14 23:11:01 2018 +0100
linux-modules: Define and use a module name database.
* gnu/build/linux-modules.scm (module-formal-name): New procedure.
(load-linux-modules-from-directory)[lookup-module]: Remove.
[module-name->file-name]: New variable. Use it.
(module-name->file-name/guess, module-name-lookup)
(write-module-name-database): New procedures.
* gnu/system/linux-initrd.scm (flat-linux-module-directory): Call
'write-module-name-database'.
diff --git a/gnu/build/linux-modules.scm b/gnu/build/linux-modules.scm
index 615e9e1d07..7e99f62941 100644
--- a/gnu/build/linux-modules.scm
+++ b/gnu/build/linux-modules.scm
@@ -31,8 +31,10 @@
#:use-module (ice-9 match)
#:use-module (ice-9 rdelim)
#:use-module (ice-9 ftw)
+ #:autoload (ice-9 pretty-print) (pretty-print)
#:export (dot-ko
ensure-dot-ko
+ module-formal-name
module-aliases
module-dependencies
recursive-module-dependencies
@@ -46,6 +48,7 @@
device-module-aliases
known-module-aliases
matching-modules
+ write-module-name-database
write-module-alias-database
write-module-device-database
load-needed-linux-modules))
@@ -95,6 +98,14 @@ key/value pairs.."
(define %not-comma
(char-set-complement (char-set #\,)))
+(define (module-formal-name file)
+ "Return the module name of FILE as it appears in its info section. Usually
+the module name is the same as the base name of FILE, modulo hyphens and minus
+the \".ko\" extension."
+ (match (assq 'name (modinfo-section-contents file))
+ (('name . name) name)
+ (#f #f)))
+
(define (module-dependencies file)
"Return the list of modules that FILE depends on. The returned list
contains module names, not actual file names."
@@ -240,12 +251,13 @@ appears in BLACK-LIST are not loaded."
"Load MODULES and their dependencies from DIRECTORY, a directory containing
the '.ko' files. The '.ko' suffix is automatically added to MODULES if
needed."
- (define (lookup-module name)
- (string-append directory "/" (ensure-dot-ko name)))
+ (define module-name->file-name
+ (module-name-lookup directory))
- (for-each (cut load-linux-module* <>
- #:lookup-module lookup-module)
- (map lookup-module modules)))
+ (for-each (lambda (module)
+ (load-linux-module* (module-name->file-name module)
+ #:lookup-module module-name->file-name))
+ modules))
;;;
@@ -401,6 +413,47 @@ ALIAS is a string like \"scsi:t-0x00\" as returned by
module)))
known-aliases))
+(define (module-name->file-name/guess directory name)
+ "Guess the file name corresponding to NAME, a module name. That doesn't
+always work because sometimes underscores in NAME map to hyphens (e.g.,
+\"input-leds.ko\"), sometimes not (e.g., \"mac_hid.ko\")."
+ (string-append directory "/" (ensure-dot-ko name)))
+
+(define (module-name-lookup directory)
+ "Return a one argument procedure that takes a module name (e.g.,
+\"input_leds\") and returns its absolute file name (e.g.,
+\"/.../input-leds.ko\")."
+ (catch 'system-error
+ (lambda ()
+ (define mapping
+ (call-with-input-file (string-append directory "/modules.name")
+ read))
+
+ (lambda (name)
+ (or (assoc-ref mapping name)
+ (module-name->file-name/guess directory name))))
+ (lambda args
+ (if (= ENOENT (system-error-errno args))
+ (cut module-name->file-name/guess directory <>)
+ (apply throw args)))))
+
+(define (write-module-name-database directory)
+ "Write a database that maps \"module names\" as they appear in the relevant
+ELF section of '.ko' files, to actual file names. This format is
+Guix-specific. It aims to deal with inconsistent naming, in particular
+hyphens vs. underscores."
+ (define mapping
+ (map (lambda (file)
+ (match (module-formal-name file)
+ (#f (cons (basename file ".ko") file))
+ (name (cons name file))))
+ (find-files directory "\\.ko$")))
+
+ (call-with-output-file (string-append directory "/modules.name")
+ (lambda (port)
+ (display ";; Module name to file name mapping.\n" port)
+ (pretty-print mapping port))))
+
(define (write-module-alias-database directory)
"Traverse the '.ko' files in DIRECTORY and create the corresponding
'modules.alias' file."
diff --git a/gnu/system/linux-initrd.scm b/gnu/system/linux-initrd.scm
index 4379fb461e..9160d94847 100644
--- a/gnu/system/linux-initrd.scm
+++ b/gnu/system/linux-initrd.scm
@@ -242,6 +242,9 @@ MODULES and taken from LINUX."
(write-module-alias-database #$output)
(write-module-device-database #$output)
+ ;; Hyphen or underscore? This database tells us.
+ (write-module-name-database #$output)
+
;; Copy "modules.builtin", which we need to determine whether a
;; module needs to be loaded.
(match (find-files #$linux (string->regexp "modules.builtin"))
Information forwarded
to
bug-guix <at> gnu.org
:
bug#34902
; Package
guix
.
(Thu, 04 Apr 2019 20:43:02 GMT)
Full text and
rfc822 format available.
Message #20 received at 34902 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Maybe I'm too paranoid but can we have "guix" in the file name "modules.name"
somewhere? Otherwise I see it coming that upstream uses modules.name for an
incompatible purpose and then we'd be with a guix interface that's broken
and/or break their interface.
(So much complexity for something so silly. Honestly, I feel like E-mailing
the upstream author and telling him what I think. WTF :P)
Should we warn when we use the fallback? I like the defensive programming
but I feel we shouldn't have it *silently* fall back when the database is
broken/missing.
Otherwise LGTM!
>(basename file ".ko")
^^^^^^
Nice. Where was that all my life. ;-)
[Message part 2 (application/pgp-signature, inline)]
Merged 34902 36574.
Request was from
Meiyo Peng <meiyo <at> riseup.net>
to
control <at> debbugs.gnu.org
.
(Thu, 11 Jul 2019 01:47:01 GMT)
Full text and
rfc822 format available.
Information forwarded
to
bug-guix <at> gnu.org
:
bug#34902
; Package
guix
.
(Thu, 01 Aug 2019 04:55:02 GMT)
Full text and
rfc822 format available.
Message #25 received at 34902 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
I have the same issue with sdhci_acpi > sdhci-acpi (See attached image) on my Laptop.
Please fix this somehow, I'm a noob and don't know how to fix that outside of the installer.
[Message part 2 (text/html, inline)]
[IMG_20190801_013919.jpg (image/jpeg, attachment)]
Information forwarded
to
bug-guix <at> gnu.org
:
bug#34902
; Package
guix
.
(Fri, 16 Aug 2019 16:58:01 GMT)
Full text and
rfc822 format available.
Message #28 received at 34902 <at> debbugs.gnu.org (full text, mbox):
Hi Ludo,
Your patch has a LGTM, but I don't see it on master. Would you like to push it, or is there a reason why you didn't do it yet?
Thanks!
Changed bug title to 'Kernel module not found at boot time due to hyphen/underscore mismatch' from 'guix cannot find a module on boot'
Request was from
Ludovic Courtès <ludo <at> gnu.org>
to
control <at> debbugs.gnu.org
.
(Fri, 16 Aug 2019 21:06:02 GMT)
Full text and
rfc822 format available.
Severity set to 'important' from 'normal'
Request was from
Ludovic Courtès <ludo <at> gnu.org>
to
control <at> debbugs.gnu.org
.
(Fri, 16 Aug 2019 21:06:03 GMT)
Full text and
rfc822 format available.
Reply sent
to
Ludovic Courtès <ludo <at> gnu.org>
:
You have taken responsibility.
(Fri, 16 Aug 2019 21:11:03 GMT)
Full text and
rfc822 format available.
Notification sent
to
Julien Lepiller <julien <at> lepiller.eu>
:
bug acknowledged by developer.
(Fri, 16 Aug 2019 21:11:03 GMT)
Full text and
rfc822 format available.
Message #37 received at 34902-done <at> debbugs.gnu.org (full text, mbox):
Hello!
Danny Milosavljevic <dannym <at> scratchpost.org> skribis:
> Maybe I'm too paranoid but can we have "guix" in the file name "modules.name"
> somewhere? Otherwise I see it coming that upstream uses modules.name for an
> incompatible purpose and then we'd be with a guix interface that's broken
> and/or break their interface.
>
> (So much complexity for something so silly. Honestly, I feel like E-mailing
> the upstream author and telling him what I think. WTF :P)
>
> Should we warn when we use the fallback? I like the defensive programming
> but I feel we shouldn't have it *silently* fall back when the database is
> broken/missing.
>
> Otherwise LGTM!
So I went ahead and pushed these patches, derived from our beautiful
patch set at <https://issues.guix.gnu.org/issue/30604>:
c85ccf60bf linux-modules: Define and use a module name database.
e1a9a7f275 linux-modules: Add 'load-linux-modules-from-directory'.
2a693b69ca linux-modules: Add "modules.devname" writer.
4f8b9d1a6f linux-modules: Add "modules.alias" writer.
The actual fix for the hyphen/underscore mismatch that Julien reported
is commit c85ccf60bf. The “modules.devname” and “modules.alias” are
actually unused so far but (1) it was easier to preserve them, and (2)
that’ll give us an incentive to finish
<https://issues.guix.gnu.org/issue/30604>. :-)
I added an explicit comment that “modules.name” uses a Guix-specific
format. We can always rename it if the kernel folks decide to acquire
that file name.
Julien, could you please confirm that your initial issue is fixed?
Thanks,
Ludo’.
Reply sent
to
Ludovic Courtès <ludo <at> gnu.org>
:
You have taken responsibility.
(Fri, 16 Aug 2019 21:11:03 GMT)
Full text and
rfc822 format available.
Notification sent
to
Meiyo Peng <meiyo <at> riseup.net>
:
bug acknowledged by developer.
(Fri, 16 Aug 2019 21:11:04 GMT)
Full text and
rfc822 format available.
bug archived.
Request was from
Debbugs Internal Request <help-debbugs <at> gnu.org>
to
internal_control <at> debbugs.gnu.org
.
(Sat, 14 Sep 2019 11:24:03 GMT)
Full text and
rfc822 format available.
This bug report was last modified 4 years and 232 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.