GNU bug report logs - #29773
urandom-seed-service should run earlier in the boot process

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 29773 in the body.
You can then email your comments to 29773 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: bug-guix <at> gnu.org
Subject: urandom-seed-service should run earlier in the boot process
Date: Tue, 19 Dec 2017 14:13:48 -0500

[Message part 1 (text/plain, inline)]

The urandom-seed-service preserves some of the Linux cryptographic
random number generator's (CRNG) pool across reboots so that freshly
booted systems have access to unpredictable psuedorandom numbers. It
also tries to take advantage of whatever is backing /dev/hwrng.

However, the urandom-seed-service may not be started before certain
applications that assume a good source of randomness.

In some cases, the applications require some random data before any
services are started, during activation. For example, our OpenSSH
service generates its host keys during activation. And even if it
generated host keys during the start of the OpenSSH service, that
service does not depend on urandom-seed-service. [0]

In systemd, there is an abstract sysinit "target" that basically serves
as a checkpoint. All the lower-level system initialization is required
before the sysinit.target is met, and the rest of the services depend on
sysinit. The random seeding is part of sysinit. I've reproduced a graph
of this in [1].

In practice, I'm not sure if it matters. I'd appreciate if GuixSD users
could check /var/log/messages for warnings like this one and report
them:

random: application: uninitialized urandom read (16 bytes read) 

And I'd also appreciate any feedback or ideas for improvement in this
area.

[0] See the attached service graph of a bare-bones system with OpenSSH.
[1] from: <https://www.freedesktop.org/software/systemd/man/bootup.html>

local-fs-pre.target
         |
         v
(various mounts and   (various swap   (various cryptsetup
 fsck services...)     devices...)        devices...)       (various low-level   (various low-level
         |                  |                  |             services: udevd,     API VFS mounts:
         v                  v                  v             tmpfiles, random     mqueue, configfs,
  local-fs.target      swap.target     cryptsetup.target    seed, sysctl, ...)      debugfs, ...)
         |                  |                  |                    |                    |
         \__________________|_________________ | ___________________|____________________/
                                              \|/
                                               v
                                        sysinit.target
                                               |
          ____________________________________/|\________________________________________
         /                  |                  |                    |                    \
         |                  |                  |                    |                    |
         v                  v                  |                    v                    v
     (various           (various               |                (various          rescue.service
    timers...)          paths...)              |               sockets...)               |
         |                  |                  |                    |                    v
         v                  v                  |                    v              rescue.target
   timers.target      paths.target             |             sockets.target
         |                  |                  |                    |
         v                  \_________________ | ___________________/
                                              \|/
                                               v
                                         basic.target
                                               |
          ____________________________________/|                                 emergency.service
         /                  |                  |                                         |
         |                  |                  |                                         v
         v                  v                  v                                 emergency.target
     display-        (various system    (various system
 manager.service         services           services)
         |             required for            |
         |            graphical UIs)           v
         |                  |           multi-user.target
         |                  |                  |
         \_________________ | _________________/
                           \|/
                            v
                  graphical.target

[signature.asc (application/pgp-signature, inline)]

Message #8 received at 29773 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: 29773 <at> debbugs.gnu.org
Subject: Service graph
Date: Tue, 19 Dec 2017 14:24:52 -0500

[Message part 1 (text/plain, inline)]

I forgot to attach the GuixSD service graph used an example in my last
message. 

Here it is!

[service-graph.svg (image/svg+xml, attachment)]

[signature.asc (application/pgp-signature, inline)]

Message #11 received at 29773 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Leo Famulari <leo <at> famulari.name>
Cc: 29773 <at> debbugs.gnu.org
Subject: Re: bug#29773: urandom-seed-service should run earlier in the boot
 process
Date: Wed, 20 Dec 2017 11:19:36 +0100

[Message part 1 (text/plain, inline)]

Hello,

Leo Famulari <leo <at> famulari.name> skribis:

> In some cases, the applications require some random data before any
> services are started, during activation. For example, our OpenSSH
> service generates its host keys during activation. And even if it
> generated host keys during the start of the OpenSSH service, that
> service does not depend on urandom-seed-service. [0]
>
> In systemd, there is an abstract sysinit "target" that basically serves
> as a checkpoint. All the lower-level system initialization is required
> before the sysinit.target is met, and the rest of the services depend on
> sysinit. The random seeding is part of sysinit. I've reproduced a graph
> of this in [1].

There’s a ‘user-processes’ service that serves a similar purpose.

With the attached patches ‘urandom-seed’ becomes a dependency of
‘user-processes’, meaning that daemons & co. start after
‘urandom-seed’.

WDYT?

> In practice, I'm not sure if it matters. I'd appreciate if GuixSD users
> could check /var/log/messages for warnings like this one and report
> them:
>
> random: application: uninitialized urandom read (16 bytes read) 

I don’t have any of these.  I guess this is most likely to happen when
running ‘ssh-keygen’ on startup, which isn’t the case on my machine.

Ludo’.

[0002-services-urandom-seed-Become-a-dependency-of-user-pr.patch (text/x-patch, attachment)]

[0001-services-user-processes-service-type-can-now-be-exte.patch (text/x-patch, attachment)]

Message #16 received at 29773 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 29773 <at> debbugs.gnu.org
Subject: Re: bug#29773: urandom-seed-service should run earlier in the boot
 process
Date: Wed, 20 Dec 2017 18:07:51 -0500

[Message part 1 (text/plain, inline)]

On Wed, Dec 20, 2017 at 11:19:36AM +0100, Ludovic Courtès wrote:
> There’s a ‘user-processes’ service that serves a similar purpose.
> 
> With the attached patches ‘urandom-seed’ becomes a dependency of
> ‘user-processes’, meaning that daemons & co. start after
> ‘urandom-seed’.
> 
> WDYT?

In general, I think it's a good approach.

Currently, the urandom-seed-service seems to non-deterministically but
typically start after the udev-service, so that /dev/hwrng is always set
up by udev before the urandom-seed-service tries to use it.

With these patches, that's not the case. This breaks the hwrng seeding
feature added in 9a56cf2b5b (services: urandom-seed: Try using a HWRNG
to seed the Linux CRNG at boot).

I'll try rearranging the service dependency graph.

> > Leo Famulari <leo <at> famulari.name> skribis:
> > In practice, I'm not sure if it matters. I'd appreciate if GuixSD users
> > could check /var/log/messages for warnings like this one and report
> > them:
> >
> > random: application: uninitialized urandom read (16 bytes read) 
> 
> I don’t have any of these.  I guess this is most likely to happen when
> running ‘ssh-keygen’ on startup, which isn’t the case on my machine.

Watching a fresh system boot repeatedly, I noticed that the host keys
always seem to be generated immediately after Linux reports "random:
crng init done".

To me, this suggests that OpenSSH is using the getrandom() syscall. If
so, any GuixSD host keys created with glibc >= 2.25 and OpenSSH >= 7.2
should be unpredictable. But I'm not sure if that's what's happening or
not.

> +(define (user-processes-shepherd-service requirements)
> +  "Return the 'user-processes' Shepherd service with dependencies on
> +REQUIREMENTS (a list of service names).
> +
> +This is a synchronization point used to make sure user processes and daemons
> +get started only after crucial initial services have been started---file
> +system mounts, etc.  This is similar to 'target' in systemd."

To clarify, user-processes may be similar to the sysinit target in
systemd. Systemd targets are sort of like run-levels, and there are
several of them, such as the multi-user target, the graphical target,
etc.

[signature.asc (application/pgp-signature, inline)]

Message #19 received at 29773 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Leo Famulari <leo <at> famulari.name>
Cc: 29773 <at> debbugs.gnu.org
Subject: Re: bug#29773: urandom-seed-service should run earlier in the boot
 process
Date: Thu, 21 Dec 2017 10:10:29 +0100

[Message part 1 (text/plain, inline)]

Leo Famulari <leo <at> famulari.name> skribis:

> On Wed, Dec 20, 2017 at 11:19:36AM +0100, Ludovic Courtès wrote:
>> There’s a ‘user-processes’ service that serves a similar purpose.
>> 
>> With the attached patches ‘urandom-seed’ becomes a dependency of
>> ‘user-processes’, meaning that daemons & co. start after
>> ‘urandom-seed’.
>> 
>> WDYT?
>
> In general, I think it's a good approach.
>
> Currently, the urandom-seed-service seems to non-deterministically but
> typically start after the udev-service, so that /dev/hwrng is always set
> up by udev before the urandom-seed-service tries to use it.
>
> With these patches, that's not the case. This breaks the hwrng seeding
> feature added in 9a56cf2b5b (services: urandom-seed: Try using a HWRNG
> to seed the Linux CRNG at boot).
>
> I'll try rearranging the service dependency graph.

The attached patch does the trick, AFAICS:

[Message part 2 (text/x-patch, inline)]

diff --git a/gnu/services/base.scm b/gnu/services/base.scm
index acc5c33f5..7fc8f6aa7 100644
--- a/gnu/services/base.scm
+++ b/gnu/services/base.scm
@@ -529,7 +529,10 @@ in KNOWN-MOUNT-POINTS when it is stopped."
   (list (shepherd-service
          (documentation "Preserve entropy across reboots for /dev/urandom.")
          (provision '(urandom-seed))
-         (requirement '(file-systems))
+
+         ;; Depend on udev so that /dev/hwrng is available.
+         (requirement '(file-systems udev))
+
          (start #~(lambda _
                     ;; On boot, write random seed into /dev/urandom.
                     (when (file-exists? #$%random-seed-file)

[Message part 3 (text/plain, inline)]

> Watching a fresh system boot repeatedly, I noticed that the host keys
> always seem to be generated immediately after Linux reports "random:
> crng init done".
>
> To me, this suggests that OpenSSH is using the getrandom() syscall. If
> so, any GuixSD host keys created with glibc >= 2.25 and OpenSSH >= 7.2
> should be unpredictable. But I'm not sure if that's what's happening or
> not.

Nice.

The problem though is that ‘ssh-keygen -A’ runs from the activation
snippet, which itself runs before shepherd is started.

To work around that, we should either introduce a separate ‘ssh-keygen’
service that ‘ssh-daemon’ would depend on, or invoke ‘ssh-keygen’ from
the ‘start’ method of the ‘ssh-daemon’ service.

>> +(define (user-processes-shepherd-service requirements)
>> +  "Return the 'user-processes' Shepherd service with dependencies on
>> +REQUIREMENTS (a list of service names).
>> +
>> +This is a synchronization point used to make sure user processes and daemons
>> +get started only after crucial initial services have been started---file
>> +system mounts, etc.  This is similar to 'target' in systemd."
>
> To clarify, user-processes may be similar to the sysinit target in
> systemd. Systemd targets are sort of like run-levels, and there are
> several of them, such as the multi-user target, the graphical target,
> etc.

Indeed, I’ve fixed it locally.

If that’s OK I’ll push these patches later today.

Thank you,
Ludo’.

Message #22 received at 29773 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 29773 <at> debbugs.gnu.org
Subject: Re: bug#29773: urandom-seed-service should run earlier in the boot
 process
Date: Thu, 21 Dec 2017 14:09:14 -0500

[Message part 1 (text/plain, inline)]

On Thu, Dec 21, 2017 at 10:10:29AM +0100, Ludovic Courtès wrote:
> 
> The attached patch does the trick, AFAICS:
> 

> diff --git a/gnu/services/base.scm b/gnu/services/base.scm
> index acc5c33f5..7fc8f6aa7 100644
> --- a/gnu/services/base.scm
> +++ b/gnu/services/base.scm
> @@ -529,7 +529,10 @@ in KNOWN-MOUNT-POINTS when it is stopped."
>    (list (shepherd-service
>           (documentation "Preserve entropy across reboots for /dev/urandom.")
>           (provision '(urandom-seed))
> -         (requirement '(file-systems))
> +
> +         ;; Depend on udev so that /dev/hwrng is available.
> +         (requirement '(file-systems udev))
> +
>           (start #~(lambda _
>                      ;; On boot, write random seed into /dev/urandom.
>                      (when (file-exists? #$%random-seed-file)

Yes, it seems to work for me.

I'm unsure if the stop action of urandom-seed-service is being executed
on shutdown.

I added some print statements and sleep delays to the stop action but
the system halts faster than I expected and I don't see any writes to
/var/lib/random-seed.

[signature.asc (application/pgp-signature, inline)]

Message #27 received at 29773-done <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Leo Famulari <leo <at> famulari.name>
Cc: 29773-done <at> debbugs.gnu.org
Subject: Re: bug#29773: urandom-seed-service should run earlier in the boot
 process
Date: Fri, 22 Dec 2017 10:06:46 +0100

Hello,

Leo Famulari <leo <at> famulari.name> skribis:

> On Thu, Dec 21, 2017 at 10:10:29AM +0100, Ludovic Courtès wrote:
>> 
>> The attached patch does the trick, AFAICS:
>> 
>
>> diff --git a/gnu/services/base.scm b/gnu/services/base.scm
>> index acc5c33f5..7fc8f6aa7 100644
>> --- a/gnu/services/base.scm
>> +++ b/gnu/services/base.scm
>> @@ -529,7 +529,10 @@ in KNOWN-MOUNT-POINTS when it is stopped."
>>    (list (shepherd-service
>>           (documentation "Preserve entropy across reboots for /dev/urandom.")
>>           (provision '(urandom-seed))
>> -         (requirement '(file-systems))
>> +
>> +         ;; Depend on udev so that /dev/hwrng is available.
>> +         (requirement '(file-systems udev))
>> +
>>           (start #~(lambda _
>>                      ;; On boot, write random seed into /dev/urandom.
>>                      (when (file-exists? #$%random-seed-file)
>
> Yes, it seems to work for me.

Great, I’ve pushed the whole series.

> I'm unsure if the stop action of urandom-seed-service is being executed
> on shutdown.
>
> I added some print statements and sleep delays to the stop action but
> the system halts faster than I expected and I don't see any writes to
> /var/lib/random-seed.

/var/lib/random-seed is definitely being updated on shutdown on my
system, as can be seen from its mtime.

Thanks,
Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.