Package: guix-patches;
Reported by: Marius Bakke <marius <at> gnu.org>
Date: Wed, 8 Jul 2020 10:10:01 UTC
Severity: normal
Tags: patch
Done: Marius Bakke <marius <at> gnu.org>
Bug is archived. No further changes may be made.
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 42261 in the body.
You can then email your comments to 42261 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
View this report as an mbox folder, status mbox, maintainer mbox
guix-patches <at> gnu.org:bug#42261; Package guix-patches.
(Wed, 08 Jul 2020 10:10:02 GMT) Full text and rfc822 format available.Marius Bakke <marius <at> gnu.org>:guix-patches <at> gnu.org.
(Wed, 08 Jul 2020 10:10:02 GMT) Full text and rfc822 format available.Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
From: Marius Bakke <marius <at> gnu.org> To: guix-patches <at> gnu.org Subject: [PATCH 0/4] Add Ganeti Date: Wed, 8 Jul 2020 12:08:26 +0200
Here it comes! The much-rumoured Ganeti service, along with a draft blog post (sent as patch 5/4). Before pushing I'm going to update Ganeti to get rid of some patches, and publish an improved ganeti-instance-guix. I will also try to provision a cluster from scratch using the blog post instructions. Note: the 'ganeti-shepherd-master-failover' patch relies on a Shepherd service that has since been removed. I intend to replace it with a "force-start" action on the ganeti-wconfd service, that passes the required command-line arguments via environment variables, but other ideas to temporarily start a daemon with special parameters welcome. I have a lot going on currently and might not be able to finish this until next week. Meanwhile, feedback appreciated as always. Marius Bakke (4): gnu: Add ganeti. gnu: Add ganeti-instance-guix. gnu: Add ganeti-instance-debootstrap. services: Add ganeti. doc/guix.texi | 560 +++++++++++ gnu/local.mk | 8 + gnu/packages/patches/ganeti-copy-hmac.patch | 83 ++ .../ganeti-disable-version-symlinks.patch | 136 +++ gnu/packages/patches/ganeti-drbd-compat.patch | 168 ++++ .../patches/ganeti-haskell-pythondir.patch | 66 ++ .../ganeti-openvswitch-may-exist.patch | 25 + .../patches/ganeti-preserve-PYTHONPATH.patch | 21 + .../ganeti-shepherd-master-failover.patch | 26 + .../patches/ganeti-shepherd-support.patch | 87 ++ gnu/packages/virtualization.scm | 500 ++++++++++ gnu/services/virtualization.scm | 906 +++++++++++++++++- gnu/tests/virtualization.scm | 175 +++- 13 files changed, 2759 insertions(+), 2 deletions(-) create mode 100644 gnu/packages/patches/ganeti-copy-hmac.patch create mode 100644 gnu/packages/patches/ganeti-disable-version-symlinks.patch create mode 100644 gnu/packages/patches/ganeti-drbd-compat.patch create mode 100644 gnu/packages/patches/ganeti-haskell-pythondir.patch create mode 100644 gnu/packages/patches/ganeti-openvswitch-may-exist.patch create mode 100644 gnu/packages/patches/ganeti-preserve-PYTHONPATH.patch create mode 100644 gnu/packages/patches/ganeti-shepherd-master-failover.patch create mode 100644 gnu/packages/patches/ganeti-shepherd-support.patch -- 2.27.0
guix-patches <at> gnu.org:bug#42261; Package guix-patches.
(Wed, 08 Jul 2020 10:12:01 GMT) Full text and rfc822 format available.Message #8 received at 42261 <at> debbugs.gnu.org (full text, mbox):
From: Marius Bakke <marius <at> gnu.org> To: 42261 <at> debbugs.gnu.org Subject: [PATCH 2/4] gnu: Add ganeti-instance-guix. Date: Wed, 8 Jul 2020 12:11:15 +0200
* gnu/packages/virtualization.scm (ganeti-instance-guix): New public variable.
---
gnu/packages/virtualization.scm | 27 +++++++++++++++++++++++++++
1 file changed, 27 insertions(+)
diff --git a/gnu/packages/virtualization.scm b/gnu/packages/virtualization.scm
index 3ef9b8470a..3e3e8e3505 100644
--- a/gnu/packages/virtualization.scm
+++ b/gnu/packages/virtualization.scm
@@ -727,6 +727,33 @@ to provide fast and simple recovery after physical failures, using
commodity hardware.")
(license license:bsd-2)))
+(define-public ganeti-instance-guix
+ (package
+ (name "ganeti-instance-guix")
+ (version "0.3.1")
+ (home-page "https://github.com/mbakke/ganeti-instance-guix")
+ (source (origin
+ (method url-fetch)
+ (uri (string-append "https://github.com/mbakke/ganeti-instance-guix"
+ "/releases/download/" version
+ "/ganeti-instance-guix-" version ".tar.lz"))
+ (sha256
+ (base32
+ "0zvy143k7hp63nc5njbn2cvdhfcmsvqsaj0q2jzax8bibag2s2a8"))))
+ (build-system gnu-build-system)
+ (arguments
+ '(#:configure-flags '("--localstatedir=/var")))
+ (native-inputs
+ `(("lzip" ,lzip)))
+ (inputs
+ `(("util-linux" ,util-linux)
+ ("qemu-img" ,qemu-minimal)))
+ (synopsis "Guix OS integration for Ganeti")
+ (description
+ "This package provides a guest OS definition for Ganeti that can
+create virtual machines using Guix and an optional configuration file.")
+ (license license:gpl3+)))
+
(define-public libosinfo
(package
(name "libosinfo")
--
2.27.0
guix-patches <at> gnu.org:bug#42261; Package guix-patches.
(Wed, 08 Jul 2020 10:12:02 GMT) Full text and rfc822 format available.Message #11 received at 42261 <at> debbugs.gnu.org (full text, mbox):
From: Marius Bakke <marius <at> gnu.org> To: 42261 <at> debbugs.gnu.org Subject: [PATCH 3/4] gnu: Add ganeti-instance-debootstrap. Date: Wed, 8 Jul 2020 12:11:16 +0200
* gnu/packages/virtualization.scm (ganeti-instance-debootstrap): New public variable.
---
gnu/packages/virtualization.scm | 95 +++++++++++++++++++++++++++++++++
1 file changed, 95 insertions(+)
diff --git a/gnu/packages/virtualization.scm b/gnu/packages/virtualization.scm
index 3e3e8e3505..5a32577d91 100644
--- a/gnu/packages/virtualization.scm
+++ b/gnu/packages/virtualization.scm
@@ -46,6 +46,7 @@
#:use-module (gnu packages cross-base)
#:use-module (gnu packages curl)
#:use-module (gnu packages cyrus-sasl)
+ #:use-module (gnu packages debian)
#:use-module (gnu packages disk)
#:use-module (gnu packages dns)
#:use-module (gnu packages docbook)
@@ -754,6 +755,100 @@ commodity hardware.")
create virtual machines using Guix and an optional configuration file.")
(license license:gpl3+)))
+(define-public ganeti-instance-debootstrap
+ (package
+ (name "ganeti-instance-debootstrap")
+ ;; We need two commits on top of the latest release for compatibility
+ ;; with newer sfdisk, as well as gnt-network integration.
+ (version "0.16-2-ge145396")
+ (home-page "https://github.com/ganeti/instance-debootstrap")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference (url home-page) (commit version)))
+ (sha256
+ (base32
+ "0f2isw9d8lawzj21rrq1q9xhq8xfa65rqbhqmrn59z201x9q1336"))))
+ (build-system gnu-build-system)
+ (arguments
+ '(#:configure-flags '("--sysconfdir=/etc" "--localstatedir=/var")
+ #:phases (modify-phases %standard-phases
+ (add-after 'unpack 'add-absolute-references
+ (lambda _
+ (substitute* "common.sh.in"
+ (("/sbin/blkid") (which "blkid"))
+ (("kpartx -")
+ (string-append (which "kpartx") " -")))
+ (substitute* "import"
+ (("restore -r")
+ (string-append (which "restore") " -r")))
+ (substitute* "export"
+ (("dump -0")
+ (string-append (which "dump") " -0")))
+ (substitute* "create"
+ (("debootstrap") (which "debootstrap"))
+ (("`which run-parts`") (which "run-parts"))
+ ;; Here we actually need to hard code /bin/passwd
+ ;; because it's called via chroot, which fails if
+ ;; "/bin" is not in PATH.
+ (("passwd") "/bin/passwd"))
+ #t))
+ (add-after 'unpack 'set-dpkg-arch
+ (lambda* (#:key system #:allow-other-keys)
+ ;; The create script passes --arch to debootstrap,
+ ;; and defaults to `dpkg --print-architecture` when
+ ;; ARCH is not set in variant.conf. Hard code the
+ ;; build-time architecture to avoid the dpkg dependency.
+ (let ((dpkg-arch
+ (cond ((string-prefix? "x86_64" system)
+ "amd64")
+ ((string-prefix? "i686" system)
+ "i386")
+ ((string-prefix? "aarch64" system)
+ "arm64")
+ (else (car (string-split system #\-))))))
+ (substitute* "create"
+ (("`dpkg --print-architecture`")
+ dpkg-arch))
+ #t)))
+ (add-after 'configure 'adjust-Makefile
+ (lambda _
+ ;; Do not attempt to create /etc/ganeti/instance-debootstrap
+ ;; and /etc/default/ganeti-instance-debootstrap during install.
+ ;; They are created by the Ganeti service.
+ (substitute* "Makefile"
+ (("\\$\\(variantsdir\\)")
+ "$(prefix)/etc/ganeti/instance-debootstrap/variants")
+ (("\\$\\(defaultsdir\\)")
+ "$(prefix)/etc/default/ganeti-instance-debootstrap"))
+ #t))
+ (add-after 'install 'make-variants.list-symlink
+ (lambda* (#:key outputs #:allow-other-keys)
+ ;; The Ganeti OS API mandates a variants.list file that
+ ;; describes all supported "variants" of this OS.
+ ;; Guix generates this file, so make the original file
+ ;; a symlink to it.
+ (with-directory-excursion (string-append
+ (assoc-ref outputs "out")
+ "/share/ganeti/os/debootstrap")
+ (delete-file "variants.list")
+ (symlink "/etc/ganeti/instance-debootstrap/variants/variants.list"
+ "variants.list"))
+ #t)))))
+ (native-inputs
+ `(("autoconf" ,autoconf)
+ ("automake" ,automake)))
+ (inputs
+ `(("debianutils" ,debianutils)
+ ("debootstrap" ,debootstrap)
+ ("dump" ,dump)
+ ("kpartx" ,multipath-tools)
+ ("util-linux" ,util-linux)))
+ (synopsis "Debian OS integration for Ganeti")
+ (description
+ "This package provides a guest OS definition for Ganeti. It installs
+Debian or a derivative using @command{debootstrap}.")
+ (license license:gpl2+)))
+
(define-public libosinfo
(package
(name "libosinfo")
--
2.27.0
guix-patches <at> gnu.org:bug#42261; Package guix-patches.
(Wed, 08 Jul 2020 10:13:01 GMT) Full text and rfc822 format available.Message #14 received at 42261 <at> debbugs.gnu.org (full text, mbox):
From: Marius Bakke <marius <at> gnu.org> To: 42261 <at> debbugs.gnu.org Subject: [PATCH 1/4] gnu: Add ganeti. Date: Wed, 8 Jul 2020 12:11:14 +0200
* gnu/packages/virtualization.scm (system->qemu-target, ganeti): New variables.
* gnu/packages/patches/ganeti-copy-hmac.patch,
gnu/packages/patches/ganeti-disable-version-symlinks.patch,
gnu/packages/patches/ganeti-drbd-compat.patch,
gnu/packages/patches/ganeti-haskell-pythondir.patch,
gnu/packages/patches/ganeti-openvswitch-may-exist.patch,
gnu/packages/patches/ganeti-preserve-PYTHONPATH.patch,
gnu/packages/patches/ganeti-shepherd-master-failover.patch,
gnu/packages/patches/ganeti-shepherd-support.patch: New files.
* gnu/local.mk (dist_patch_DATA): Adjust accordingly.
---
gnu/local.mk | 8 +
gnu/packages/patches/ganeti-copy-hmac.patch | 83 ++++
.../ganeti-disable-version-symlinks.patch | 136 +++++++
gnu/packages/patches/ganeti-drbd-compat.patch | 168 ++++++++
.../patches/ganeti-haskell-pythondir.patch | 66 +++
.../ganeti-openvswitch-may-exist.patch | 25 ++
.../patches/ganeti-preserve-PYTHONPATH.patch | 21 +
.../ganeti-shepherd-master-failover.patch | 26 ++
.../patches/ganeti-shepherd-support.patch | 87 ++++
gnu/packages/virtualization.scm | 378 ++++++++++++++++++
10 files changed, 998 insertions(+)
create mode 100644 gnu/packages/patches/ganeti-copy-hmac.patch
create mode 100644 gnu/packages/patches/ganeti-disable-version-symlinks.patch
create mode 100644 gnu/packages/patches/ganeti-drbd-compat.patch
create mode 100644 gnu/packages/patches/ganeti-haskell-pythondir.patch
create mode 100644 gnu/packages/patches/ganeti-openvswitch-may-exist.patch
create mode 100644 gnu/packages/patches/ganeti-preserve-PYTHONPATH.patch
create mode 100644 gnu/packages/patches/ganeti-shepherd-master-failover.patch
create mode 100644 gnu/packages/patches/ganeti-shepherd-support.patch
diff --git a/gnu/local.mk b/gnu/local.mk
index a277e63fa4..143aae2bea 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -944,6 +944,14 @@ dist_patch_DATA = \
%D%/packages/patches/fontconfig-hurd-path-max.patch \
%D%/packages/patches/freeimage-unbundle.patch \
%D%/packages/patches/fuse-overlapping-headers.patch \
+ %D%/packages/patches/ganeti-copy-hmac.patch \
+ %D%/packages/patches/ganeti-drbd-compat.patch \
+ %D%/packages/patches/ganeti-disable-version-symlinks.patch \
+ %D%/packages/patches/ganeti-haskell-pythondir.patch \
+ %D%/packages/patches/ganeti-openvswitch-may-exist.patch \
+ %D%/packages/patches/ganeti-preserve-PYTHONPATH.patch \
+ %D%/packages/patches/ganeti-shepherd-master-failover.patch \
+ %D%/packages/patches/ganeti-shepherd-support.patch \
%D%/packages/patches/gash-utils-ls-test.patch \
%D%/packages/patches/gawk-shell.patch \
%D%/packages/patches/gcc-arm-bug-71399.patch \
diff --git a/gnu/packages/patches/ganeti-copy-hmac.patch b/gnu/packages/patches/ganeti-copy-hmac.patch
new file mode 100644
index 0000000000..c1a758afe9
--- /dev/null
+++ b/gnu/packages/patches/ganeti-copy-hmac.patch
@@ -0,0 +1,83 @@
+This is a cherry-pick of three upstream commits that got lost in the
+transition when Ganeti was donated to the community.
+
+Submitted upstream: <https://github.com/ganeti/ganeti/pull/1494>.
+
+diff --git a/lib/bootstrap.py b/lib/bootstrap.py
+--- a/lib/bootstrap.py
++++ b/lib/bootstrap.py
+@@ -934,6 +934,8 @@ def SetupNodeDaemon(opts, cluster_name, node, ssh_port):
+ constants.NDS_CLUSTER_NAME: cluster_name,
+ constants.NDS_NODE_DAEMON_CERTIFICATE:
+ utils.ReadFile(pathutils.NODED_CERT_FILE),
++ constants.NDS_HMAC:
++ utils.ReadFile(pathutils.CONFD_HMAC_KEY),
+ constants.NDS_SSCONF: ssconf.SimpleStore().ReadAll(),
+ constants.NDS_START_NODE_DAEMON: True,
+ constants.NDS_NODE_NAME: node,
+diff --git a/lib/tools/common.py b/lib/tools/common.py
+--- a/lib/tools/common.py
++++ b/lib/tools/common.py
+@@ -184,6 +184,19 @@ def VerifyClusterName(data, error_fn, cluster_name_constant,
+ return name
+
+
++def VerifyHmac(data, error_fn):
++ """Verifies the presence of the hmac secret.
++
++ @type data: dict
++
++ """
++ hmac = data.get(constants.NDS_HMAC)
++ if not hmac:
++ raise error_fn("Hmac key must be provided")
++
++ return hmac
++
++
+ def LoadData(raw, data_check):
+ """Parses and verifies input data.
+
+diff --git a/lib/tools/node_daemon_setup.py b/lib/tools/node_daemon_setup.py
+--- a/lib/tools/node_daemon_setup.py
++++ b/lib/tools/node_daemon_setup.py
+@@ -51,6 +51,7 @@ from ganeti.tools import common
+ _DATA_CHECK = ht.TStrictDict(False, True, {
+ constants.NDS_CLUSTER_NAME: ht.TNonEmptyString,
+ constants.NDS_NODE_DAEMON_CERTIFICATE: ht.TNonEmptyString,
++ constants.NDS_HMAC: ht.TNonEmptyString,
+ constants.NDS_SSCONF: ht.TDictOf(ht.TNonEmptyString, ht.TString),
+ constants.NDS_START_NODE_DAEMON: ht.TBool,
+ constants.NDS_NODE_NAME: ht.TString,
+@@ -127,11 +128,18 @@ def Main():
+ cluster_name = common.VerifyClusterName(data, SetupError,
+ constants.NDS_CLUSTER_NAME)
+ cert_pem = common.VerifyCertificateStrong(data, SetupError)
++ hmac_key = common.VerifyHmac(data, SetupError)
+ ssdata = VerifySsconf(data, cluster_name)
+
+ logging.info("Writing ssconf files ...")
+ ssconf.WriteSsconfFiles(ssdata, dry_run=opts.dry_run)
+
++ logging.info("Writing hmac.key ...")
++ utils.WriteFile(pathutils.CONFD_HMAC_KEY, data=hmac_key,
++ mode=pathutils.NODED_CERT_MODE,
++ uid=getent.masterd_uid, gid=getent.masterd_gid,
++ dry_run=opts.dry_run)
++
+ logging.info("Writing node daemon certificate ...")
+ utils.WriteFile(pathutils.NODED_CERT_FILE, data=cert_pem,
+ mode=pathutils.NODED_CERT_MODE,
+diff --git a/src/Ganeti/Constants.hs b/src/Ganeti/Constants.hs
+--- a/src/Ganeti/Constants.hs
++++ b/src/Ganeti/Constants.hs
+@@ -4833,6 +4833,9 @@ ndsNodeDaemonCertificate = "node_daemon_certificate"
+ ndsSsconf :: String
+ ndsSsconf = "ssconf"
+
++ndsHmac :: String
++ndsHmac = "hmac_key"
++
+ ndsStartNodeDaemon :: String
+ ndsStartNodeDaemon = "start_node_daemon"
+
diff --git a/gnu/packages/patches/ganeti-disable-version-symlinks.patch b/gnu/packages/patches/ganeti-disable-version-symlinks.patch
new file mode 100644
index 0000000000..a5f347cfc6
--- /dev/null
+++ b/gnu/packages/patches/ganeti-disable-version-symlinks.patch
@@ -0,0 +1,136 @@
+This patch adds a new "--disable-version-links" configuration option
+that allows installing to the standard GNU installation directories
+instead of having to add symlinks in /etc/ganeti/{lib,share} that
+points to the right $ganeti/{lib,share}/$version. Mainly to reduce
+service complexity, and because Guix users can install as many versions
+of Ganeti they can muster without resorting to such hacks.
+
+diff --git a/Makefile.am b/Makefile.am
+--- a/Makefile.am
++++ b/Makefile.am
+@@ -66,11 +66,16 @@ SHELL_ENV_INIT = autotools/shell-env-init
+ # so, if some currently architecture-independent executable is replaced by an
+ # architecture-dependent one (and hence has to go under $(versiondir)), add a link
+ # under $(versionedsharedir) but do not change the external links.
++#
++# As of Ganeti 3.0, it is possible to disable this behavior by passing
++# --disable-version-links, in which case the standard GNU installation
++# directories are used.
+ if USE_VERSION_FULL
+ DIRVERSION=$(VERSION_FULL)
+ else
+ DIRVERSION=$(VERSION_MAJOR).$(VERSION_MINOR)
+ endif
++if USE_VERSION_LINKS
+ versiondir = $(libdir)/ganeti/$(DIRVERSION)
+ defaultversiondir = $(libdir)/ganeti/default
+ versionedsharedir = $(prefix)/share/ganeti/$(DIRVERSION)
+@@ -90,6 +95,18 @@ gntpythondir = $(versionedsharedir)
+ pkgpython_bindir = $(versionedsharedir)
+ gnt_python_sbindir = $(versionedsharedir)
+ tools_pythondir = $(versionedsharedir)
++else
++myexeclibdir = $(pkglibdir)
++pkgpython_rpc_stubdir = $(pkgpythondir)/rpc/stub
++gntpythondir = $(sbindir)
++pkgpython_bindir = $(pkglibdir)
++gnt_python_sbindir = $(sbindir)
++tools_pythondir = $(pkglibdir)
++versionedsharedir = $(pkglibdir)
++# This is a hack but works because the only user does $(versiondir)$(datadir).
++versiondir =
++endif !USE_VERSION_LINKS
++
+
+ clientdir = $(pkgpythondir)/client
+ cmdlibdir = $(pkgpythondir)/cmdlib
+@@ -2356,6 +2373,7 @@ src/AutoConf.hs: Makefile src/AutoConf.hs.in $(PRINT_PY_CONSTANTS) \
+ -DVERSION_SUFFIX="$(VERSION_SUFFIX)" \
+ -DVERSION_FULL="$(VERSION_FULL)" \
+ -DDIRVERSION="$(DIRVERSION)" \
++ -DUSE_VERSION_LINKS="$(USE_VERSION_LINKS)" \
+ -DLOCALSTATEDIR="$(localstatedir)" \
+ -DSYSCONFDIR="$(sysconfdir)" \
+ -DSSH_CONFIG_DIR="$(SSH_CONFIG_DIR)" \
+@@ -2857,6 +2875,7 @@ install-exec-local:
+ @mkdir_p@ "$(DESTDIR)${localstatedir}/lib/ganeti" \
+ "$(DESTDIR)${localstatedir}/log/ganeti" \
+ "$(DESTDIR)${localstatedir}/run/ganeti"
++if USE_VERSION_LINKS
+ for dir in $(SYMLINK_TARGET_DIRS); do \
+ @mkdir_p@ $(DESTDIR)$$dir; \
+ done
+@@ -2892,7 +2911,8 @@ install-exec-local:
+ if INSTALL_SYMLINKS
+ $(LN_S) -f $(versionedsharedir) $(DESTDIR)$(sysconfdir)/ganeti/share
+ $(LN_S) -f $(versiondir) $(DESTDIR)$(sysconfdir)/ganeti/lib
+-endif
++endif INSTALL_SYMLINKS
++endif USE_VERSION_LINKS
+
+ .PHONY: apidoc
+ if WANT_HSAPIDOC
+diff --git a/configure.ac b/configure.ac
+--- a/configure.ac
++++ b/configure.ac
+@@ -29,6 +29,23 @@ AC_SUBST([BINDIR], $bindir)
+ AC_SUBST([SBINDIR], $sbindir)
+ AC_SUBST([MANDIR], $mandir)
+
++# --enable-version-links
++AC_ARG_ENABLE([version-links],
++ [AS_HELP_STRING([--enable-version-links],
++ m4_normalize([install ganeti to version-specific
++ subdirectories to allow installing multiple versions
++ in parallel (default: enabled)]))],
++ [[if test "$enableval" != no; then
++ USE_VERSION_LINKS=True
++ else
++ USE_VERSION_LINKS=False
++ fi
++ ]],
++ [USE_VERSION_LINKS=True
++ ])
++AC_SUBST(USE_VERSION_LINKS, $USE_VERSION_LINKS)
++AM_CONDITIONAL([USE_VERSION_LINKS], [test "$USE_VERSION_LINKS" = True])
++
+ # --enable-versionfull
+ AC_ARG_ENABLE([versionfull],
+ [AS_HELP_STRING([--enable-versionfull],
+diff --git a/lib/bootstrap.py b/lib/bootstrap.py
+--- a/lib/bootstrap.py
++++ b/lib/bootstrap.py
+@@ -944,7 +944,7 @@ def SetupNodeDaemon(opts, cluster_name, node, ssh_port):
+ debug=opts.debug, verbose=opts.verbose,
+ use_cluster_key=True, ask_key=opts.ssh_key_check,
+ strict_host_check=opts.ssh_key_check,
+- ensure_version=True)
++ ensure_version=constants.USE_VERSION_LINKS)
+
+ _WaitForSshDaemon(node, ssh_port)
+ _WaitForNodeDaemon(node)
+diff --git a/src/AutoConf.hs.in b/src/AutoConf.hs.in
+--- a/src/AutoConf.hs.in
++++ b/src/AutoConf.hs.in
+@@ -64,6 +64,9 @@ versionFull = "VERSION_FULL"
+ dirVersion :: String
+ dirVersion = "DIRVERSION"
+
++useVersionLinks :: Bool
++useVersionLinks = USE_VERSION_LINKS
++
+ localstatedir :: String
+ localstatedir = "LOCALSTATEDIR"
+
+diff --git a/src/Ganeti/Constants.hs b/src/Ganeti/Constants.hs
+--- a/src/Ganeti/Constants.hs
++++ b/src/Ganeti/Constants.hs
+@@ -164,5 +164,8 @@ versionRevision = AutoConf.versionRevision
+ dirVersion :: String
+ dirVersion = AutoConf.dirVersion
+
++useVersionLinks :: Bool
++useVersionLinks = AutoConf.useVersionLinks
++
+ osApiV10 :: Int
+ osApiV10 = 10
diff --git a/gnu/packages/patches/ganeti-drbd-compat.patch b/gnu/packages/patches/ganeti-drbd-compat.patch
new file mode 100644
index 0000000000..e06e04c5ef
--- /dev/null
+++ b/gnu/packages/patches/ganeti-drbd-compat.patch
@@ -0,0 +1,168 @@
+This patch adds support for newer versions of DRBD.
+
+Submitted upstream: <https://github.com/ganeti/ganeti/pull/1496>.
+
+diff --git a/lib/storage/drbd.py b/lib/storage/drbd.py
+--- a/lib/storage/drbd.py
++++ b/lib/storage/drbd.py
+@@ -315,6 +315,13 @@ class DRBD8Dev(base.BlockDev):
+ """
+ return self._show_info_cls.GetDevInfo(self._GetShowData(minor))
+
++ @staticmethod
++ def _NeedsLocalSyncerParams():
++ # For DRBD >= 8.4, syncer init must be done after local, not in net.
++ info = DRBD8.GetProcInfo()
++ version = info.GetVersion()
++ return version["k_minor"] >= 4
++
+ def _MatchesLocal(self, info):
+ """Test if our local config matches with an existing device.
+
+@@ -397,6 +404,22 @@ class DRBD8Dev(base.BlockDev):
+ base.ThrowError("drbd%d: can't attach local disk: %s",
+ minor, result.output)
+
++ def _WaitForMinorSyncParams():
++ """Call _SetMinorSyncParams and raise RetryAgain on errors.
++ """
++ if self._SetMinorSyncParams(minor, self.params):
++ raise utils.RetryAgain()
++ else:
++ return []
++
++ if self._NeedsLocalSyncerParams():
++ # Retry because disk config for DRBD resource may be still uninitialized.
++ try:
++ utils.Retry(_WaitForMinorSyncParams, 1.0, 5.0)
++ except utils.RetryTimeout:
++ base.ThrowError("drbd%d: can't set the synchronization parameters: %s" %
++ (minor, utils.CommaJoin(sync_errors)))
++
+ def _AssembleNet(self, minor, net_info, dual_pri=False, hmac=None,
+ secret=None):
+ """Configure the network part of the device.
+@@ -432,21 +455,24 @@ class DRBD8Dev(base.BlockDev):
+ # sync speed only after setting up both sides can race with DRBD
+ # connecting, hence we set it here before telling DRBD anything
+ # about its peer.
+- sync_errors = self._SetMinorSyncParams(minor, self.params)
+- if sync_errors:
+- base.ThrowError("drbd%d: can't set the synchronization parameters: %s" %
+- (minor, utils.CommaJoin(sync_errors)))
++
++ if not self._NeedsLocalSyncerParams():
++ sync_errors = self._SetMinorSyncParams(minor, self.params)
++ if sync_errors:
++ base.ThrowError("drbd%d: can't set the synchronization parameters: %s" %
++ (minor, utils.CommaJoin(sync_errors)))
+
+ family = self._GetNetFamily(minor, lhost, rhost)
+
+- cmd = self._cmd_gen.GenNetInitCmd(minor, family, lhost, lport,
++ cmds = self._cmd_gen.GenNetInitCmds(minor, family, lhost, lport,
+ rhost, rport, protocol,
+ dual_pri, hmac, secret, self.params)
+
+- result = utils.RunCmd(cmd)
+- if result.failed:
+- base.ThrowError("drbd%d: can't setup network: %s - %s",
+- minor, result.fail_reason, result.output)
++ for cmd in cmds:
++ result = utils.RunCmd(cmd)
++ if result.failed:
++ base.ThrowError("drbd%d: can't setup network: %s - %s",
++ minor, result.fail_reason, result.output)
+
+ def _CheckNetworkConfig():
+ info = self._GetShowInfo(minor)
+@@ -463,19 +489,20 @@ class DRBD8Dev(base.BlockDev):
+ base.ThrowError("drbd%d: timeout while configuring network", minor)
+
+ # Once the assembly is over, try to set the synchronization parameters
+- try:
+- # The minor may not have been set yet, requiring us to set it at least
+- # temporarily
+- old_minor = self.minor
+- self._SetFromMinor(minor)
+- sync_errors = self.SetSyncParams(self.params)
+- if sync_errors:
+- base.ThrowError("drbd%d: can't set the synchronization parameters: %s" %
+- (self.minor, utils.CommaJoin(sync_errors)))
+- finally:
+- # Undo the change, regardless of whether it will have to be done again
+- # soon
+- self._SetFromMinor(old_minor)
++ if not self._NeedsLocalSyncerParams():
++ try:
++ # The minor may not have been set yet, requiring us to set it at least
++ # temporarily
++ old_minor = self.minor
++ self._SetFromMinor(minor)
++ sync_errors = self.SetSyncParams(self.params)
++ if sync_errors:
++ base.ThrowError("drbd%d: can't set the synchronization parameters: %s" %
++ (self.minor, utils.CommaJoin(sync_errors)))
++ finally:
++ # Undo the change, regardless of whether it will have to be done again
++ # soon
++ self._SetFromMinor(old_minor)
+
+ @staticmethod
+ def _GetNetFamily(minor, lhost, rhost):
+diff --git a/lib/storage/drbd_cmdgen.py b/lib/storage/drbd_cmdgen.py
+--- a/lib/storage/drbd_cmdgen.py
++++ b/lib/storage/drbd_cmdgen.py
+@@ -56,7 +56,7 @@ class BaseDRBDCmdGenerator(object):
+ def GenLocalInitCmds(self, minor, data_dev, meta_dev, size_mb, params):
+ raise NotImplementedError
+
+- def GenNetInitCmd(self, minor, family, lhost, lport, rhost, rport, protocol,
++ def GenNetInitCmds(self, minor, family, lhost, lport, rhost, rport, protocol,
+ dual_pri, hmac, secret, params):
+ raise NotImplementedError
+
+@@ -138,7 +138,7 @@ class DRBD83CmdGenerator(BaseDRBDCmdGenerator):
+
+ return [args]
+
+- def GenNetInitCmd(self, minor, family, lhost, lport, rhost, rport, protocol,
++ def GenNetInitCmds(self, minor, family, lhost, lport, rhost, rport, protocol,
+ dual_pri, hmac, secret, params):
+ args = ["drbdsetup", self._DevPath(minor), "net",
+ "%s:%s:%s" % (family, lhost, lport),
+@@ -155,7 +155,7 @@ class DRBD83CmdGenerator(BaseDRBDCmdGenerator):
+ if params[constants.LDP_NET_CUSTOM]:
+ args.extend(shlex.split(params[constants.LDP_NET_CUSTOM]))
+
+- return args
++ return [args]
+
+ def GenSyncParamsCmd(self, minor, params):
+ args = ["drbdsetup", self._DevPath(minor), "syncer"]
+@@ -345,8 +345,14 @@ class DRBD84CmdGenerator(BaseDRBDCmdGenerator):
+
+ return cmds
+
+- def GenNetInitCmd(self, minor, family, lhost, lport, rhost, rport, protocol,
++ def GenNetInitCmds(self, minor, family, lhost, lport, rhost, rport, protocol,
+ dual_pri, hmac, secret, params):
++ cmds = []
++
++ cmds.append(["drbdsetup", "new-resource", self._GetResource(minor)])
++ cmds.append(["drbdsetup", "new-minor", self._GetResource(minor),
++ str(minor), "0"])
++
+ args = ["drbdsetup", "connect", self._GetResource(minor),
+ "%s:%s:%s" % (family, lhost, lport),
+ "%s:%s:%s" % (family, rhost, rport),
+@@ -362,7 +368,8 @@ class DRBD84CmdGenerator(BaseDRBDCmdGenerator):
+ if params[constants.LDP_NET_CUSTOM]:
+ args.extend(shlex.split(params[constants.LDP_NET_CUSTOM]))
+
+- return args
++ cmds.append(args)
++ return cmds
+
+ def GenSyncParamsCmd(self, minor, params):
+ args = ["drbdsetup", "disk-options", minor]
diff --git a/gnu/packages/patches/ganeti-haskell-pythondir.patch b/gnu/packages/patches/ganeti-haskell-pythondir.patch
new file mode 100644
index 0000000000..fa77771839
--- /dev/null
+++ b/gnu/packages/patches/ganeti-haskell-pythondir.patch
@@ -0,0 +1,66 @@
+This patch allows the Haskell daemons to locate Python libraries
+installed to a non-standard pythondir. It is necessary because Guix
+does not use versionedsharedir (see related patch that disables it).
+
+diff --git a/Makefile.am b/Makefile.am
+--- a/Makefile.am
++++ b/Makefile.am
+@@ -83,6 +83,7 @@ myexeclibdir = $(pkglibdir)
+ bindir = $(versiondir)/$(BINDIR)
+ sbindir = $(versiondir)$(SBINDIR)
+ mandir = $(versionedsharedir)/root$(MANDIR)
++pythondir = $(versionedsharedir)
+ pkgpythondir = $(versionedsharedir)/ganeti
+ pkgpython_rpc_stubdir = $(versionedsharedir)/ganeti/rpc/stub
+ gntpythondir = $(versionedsharedir)
+@@ -2386,6 +2387,7 @@ src/AutoConf.hs: Makefile src/AutoConf.hs.in $(PRINT_PY_CONSTANTS) \
+ -DPKGLIBDIR="$(libdir)/ganeti" \
+ -DSHAREDIR="$(prefix)/share/ganeti" \
+ -DVERSIONEDSHAREDIR="$(versionedsharedir)" \
++ -DPYTHONDIR="$(pythondir)" \
+ -DDRBD_BARRIERS="$(DRBD_BARRIERS)" \
+ -DDRBD_NO_META_FLUSH="$(DRBD_NO_META_FLUSH)" \
+ -DSYSLOG_USAGE="$(SYSLOG_USAGE)" \
+diff --git a/src/AutoConf.hs.in b/src/AutoConf.hs.in
+--- a/src/AutoConf.hs.in
++++ b/src/AutoConf.hs.in
+@@ -157,6 +157,9 @@ sharedir = "SHAREDIR"
+ versionedsharedir :: String
+ versionedsharedir = "VERSIONEDSHAREDIR"
+
++pythondir :: String
++pythondir = "PYTHONDIR"
++
+ drbdBarriers :: String
+ drbdBarriers = "DRBD_BARRIERS"
+
+diff --git a/src/Ganeti/Path.hs b/src/Ganeti/Path.hs
+--- a/src/Ganeti/Path.hs
++++ b/src/Ganeti/Path.hs
+@@ -188,5 +188,5 @@ getInstReasonFilename instName = instanceReasonDir `pjoin` instName
+
+ -- | The path to the Python executable for starting jobs.
+ jqueueExecutorPy :: IO FilePath
+-jqueueExecutorPy = return $ versionedsharedir
+- </> "ganeti" </> "jqueue" </> "exec.py"
++jqueueExecutorPy = return $ pythondir
++ </> "ganeti" </> "jqueue" </> "exec.py"
+diff --git a/src/Ganeti/Query/Exec.hs b/src/Ganeti/Query/Exec.hs
+--- a/src/Ganeti/Query/Exec.hs
++++ b/src/Ganeti/Query/Exec.hs
+@@ -99,12 +99,12 @@ spawnJobProcess jid = withErrorLogAt CRITICAL (show jid) $
+ do
+ use_debug <- isDebugMode
+ env_ <- (M.toList . M.insert "GNT_DEBUG" (if use_debug then "1" else "0")
+- . M.insert "PYTHONPATH" AC.versionedsharedir
++ . M.insert "PYTHONPATH" AC.pythondir
+ . M.fromList)
+ `liftM` getEnvironment
+ execPy <- P.jqueueExecutorPy
+ logDebug $ "Executing " ++ AC.pythonPath ++ " " ++ execPy
+- ++ " with PYTHONPATH=" ++ AC.versionedsharedir
++ ++ " with PYTHONPATH=" ++ AC.pythondir
+
+ (master, child) <- pipeClient connectConfig
+ let (rh, wh) = clientToHandle child
+
diff --git a/gnu/packages/patches/ganeti-openvswitch-may-exist.patch b/gnu/packages/patches/ganeti-openvswitch-may-exist.patch
new file mode 100644
index 0000000000..03543167f6
--- /dev/null
+++ b/gnu/packages/patches/ganeti-openvswitch-may-exist.patch
@@ -0,0 +1,25 @@
+It is possible that the switch is already configured by external tools.
+Do not error out when that is the case.
+
+Submitted upstream: <https://github.com/ganeti/ganeti/pull/1495>.
+
+diff --git a/lib/backend.py b/lib/backend.py
+--- a/lib/backend.py
++++ b/lib/backend.py
+@@ -5865,14 +5865,14 @@ def ConfigureOVS(ovs_name, ovs_link):
+
+ """
+ # Initialize the OpenvSwitch
+- result = utils.RunCmd(["ovs-vsctl", "add-br", ovs_name])
++ result = utils.RunCmd(["ovs-vsctl", "--may-exist", "add-br", ovs_name])
+ if result.failed:
+ _Fail("Failed to create openvswitch. Script return value: %s, output: '%s'"
+ % (result.exit_code, result.output), log=True)
+
+ # And connect it to a physical interface, if given
+ if ovs_link:
+- result = utils.RunCmd(["ovs-vsctl", "add-port", ovs_name, ovs_link])
++ result = utils.RunCmd(["ovs-vsctl", "--may-exist", "add-port", ovs_name, ovs_link])
+ if result.failed:
+ _Fail("Failed to connect openvswitch to interface %s. Script return"
+ " value: %s, output: '%s'" % (ovs_link, result.exit_code,
diff --git a/gnu/packages/patches/ganeti-preserve-PYTHONPATH.patch b/gnu/packages/patches/ganeti-preserve-PYTHONPATH.patch
new file mode 100644
index 0000000000..1358e30633
--- /dev/null
+++ b/gnu/packages/patches/ganeti-preserve-PYTHONPATH.patch
@@ -0,0 +1,21 @@
+Do not override PYTHONPATH when calling Python code from the Haskell
+daemons. This is necessary because the Python library dependencies are
+only available through PYTHONPATH.
+
+diff --git a/src/Ganeti/Query/Exec.hs b/src/Ganeti/Query/Exec.hs
+--- a/src/Ganeti/Query/Exec.hs
++++ b/src/Ganeti/Query/Exec.hs
+@@ -99,12 +99,10 @@ spawnJobProcess jid = withErrorLogAt CRITICAL (show jid) $
+ do
+ use_debug <- isDebugMode
+ env_ <- (M.toList . M.insert "GNT_DEBUG" (if use_debug then "1" else "0")
+- . M.insert "PYTHONPATH" AC.pythondir
+ . M.fromList)
+ `liftM` getEnvironment
+ execPy <- P.jqueueExecutorPy
+ logDebug $ "Executing " ++ AC.pythonPath ++ " " ++ execPy
+- ++ " with PYTHONPATH=" ++ AC.pythondir
+
+ (master, child) <- pipeClient connectConfig
+ let (rh, wh) = clientToHandle child
+
diff --git a/gnu/packages/patches/ganeti-shepherd-master-failover.patch b/gnu/packages/patches/ganeti-shepherd-master-failover.patch
new file mode 100644
index 0000000000..d255233fc1
--- /dev/null
+++ b/gnu/packages/patches/ganeti-shepherd-master-failover.patch
@@ -0,0 +1,26 @@
+By default, master-failover will call "herd start ganeti-wconfd" with
+extra arguments such as --force-node. That does not work with the
+Shepherd, so Guix has a "ganeti-wconfd-forced" service for this purpose.
+
+diff --git a/lib/bootstrap.py b/lib/bootstrap.py
+--- a/lib/bootstrap.py
++++ b/lib/bootstrap.py
+@@ -1010,8 +1010,7 @@ def MasterFailover(no_voting=False):
+ try:
+ # Forcefully start WConfd so that we can access the configuration
+ result = utils.RunCmd([pathutils.DAEMON_UTIL,
+- "start", constants.WCONFD, "--force-node",
+- "--no-voting", "--yes-do-it"])
++ "start", "ganeti-wconfd-forced"])
+ if result.failed:
+ raise errors.OpPrereqError("Could not start the configuration daemon,"
+ " command %s had exitcode %s and error %s" %
+@@ -1074,7 +1073,7 @@ def MasterFailover(no_voting=False):
+ return 1, warnings
+ finally:
+ # stop WConfd again:
+- result = utils.RunCmd([pathutils.DAEMON_UTIL, "stop", constants.WCONFD])
++ result = utils.RunCmd([pathutils.DAEMON_UTIL, "stop", "ganeti-wconfd-forced"])
+ if result.failed:
+ warning = ("Could not stop the configuration daemon,"
+ " command %s had exitcode %s and error %s"
diff --git a/gnu/packages/patches/ganeti-shepherd-support.patch b/gnu/packages/patches/ganeti-shepherd-support.patch
new file mode 100644
index 0000000000..f750604344
--- /dev/null
+++ b/gnu/packages/patches/ganeti-shepherd-support.patch
@@ -0,0 +1,87 @@
+Ganeti uses an internal tool to start/stop daemons during init and
+upgrade. This patch makes the tool use native Shepherd facilities.
+
+diff --git a/daemons/daemon-util.in b/daemons/daemon-util.in
+--- a/daemons/daemon-util.in
++++ b/daemons/daemon-util.in
+@@ -184,6 +184,21 @@ use_systemctl() {
+ return 1
+ }
+
++# Checks if we should use the Shepherd to start/stop daemons
++use_shepherd() {
++ # Is Shepherd running as PID 1?
++ ps --no-headers -p 1 -o cmd | grep -q shepherd || return 1
++
++ type -p herd >/dev/null || return 1
++
++ # Does Shepherd know about Ganeti at all?
++ if herd status | grep -q ganeti; then
++ return 0
++ fi
++
++ return 1
++}
++
+ # Prints path to PID file for a daemon.
+ daemon_pidfile() {
+ if [[ "$#" -lt 1 ]]; then
+@@ -261,6 +276,13 @@ check() {
+ else
+ return 1
+ fi
++ elif use_shepherd; then
++ activestate="$(herd status ${name})"
++ if echo $activestate | grep -q Running ; then
++ return 0
++ else
++ return 1
++ fi
+ elif type -p start-stop-daemon >/dev/null; then
+ start-stop-daemon --stop --signal 0 --quiet \
+ --pidfile $pidfile --name "$name"
+@@ -291,6 +313,20 @@ start() {
+ return $?
+ fi
+
++ if use_shepherd; then
++ if herd status "$name" | grep -q "disabled"; then
++ # The Shepherd will disable a service that has stopped, even if it exits
++ # gracefully. Thus, we must re-enable it in case of a master failover.
++ herd enable "${name}"
++ fi
++ # Note: unlike systemd, which happily starts a service and returns success
++ # even if the daemon immediately exits, the Shepherd actually waits for it
++ # to come up. Thus, ignore the exit status from 'herd start' in case of
++ # master daemons running on the wrong node, or ganeti-kvmd disabled, etc.
++ herd start "${name}"
++ return 0
++ fi
++
+ # Read $<daemon>_ARGS and $EXTRA_<daemon>_ARGS
+ eval local args="\"\$${ucname}_ARGS \$EXTRA_${ucname}_ARGS\""
+
+@@ -336,6 +372,13 @@ stop() {
+
+ if use_systemctl; then
+ systemctl stop "${name}.service"
++ elif use_shepherd; then
++ if herd status | grep -q "$name"; then
++ herd stop "$name"
++ else
++ # Do not raise an error if the service has not been enabled.
++ return 0
++ fi
+ elif type -p start-stop-daemon >/dev/null; then
+ start-stop-daemon --stop --quiet --oknodo --retry 30 \
+ --pidfile $pidfile --name "$name"
+@@ -352,6 +395,9 @@ check_and_start() {
+ if use_systemctl; then
+ echo "${name} supervised by systemd but not running, will not restart."
+ return 1
++ elif use_shepherd; then
++ echo "${name} supervised by shepherd but not running, will not restart."
++ return 1
+ fi
+
+ start $name
diff --git a/gnu/packages/virtualization.scm b/gnu/packages/virtualization.scm
index 64cd815bb8..3ef9b8470a 100644
--- a/gnu/packages/virtualization.scm
+++ b/gnu/packages/virtualization.scm
@@ -38,6 +38,7 @@
#:use-module (gnu packages attr)
#:use-module (gnu packages autotools)
#:use-module (gnu packages backup)
+ #:use-module (gnu packages base)
#:use-module (gnu packages bison)
#:use-module (gnu packages check)
#:use-module (gnu packages cmake)
@@ -61,10 +62,17 @@
#:use-module (gnu packages gnupg)
#:use-module (gnu packages golang)
#:use-module (gnu packages gtk)
+ #:use-module (gnu packages haskell)
+ #:use-module (gnu packages haskell-apps)
+ #:use-module (gnu packages haskell-check)
+ #:use-module (gnu packages haskell-crypto)
+ #:use-module (gnu packages haskell-web)
+ #:use-module (gnu packages haskell-xyz)
#:use-module (gnu packages image)
#:use-module (gnu packages libbsd)
#:use-module (gnu packages libusb)
#:use-module (gnu packages linux)
+ #:use-module (gnu packages m4)
#:use-module (gnu packages ncurses)
#:use-module (gnu packages nettle)
#:use-module (gnu packages networking)
@@ -75,6 +83,7 @@
#:use-module (gnu packages polkit)
#:use-module (gnu packages protobuf)
#:use-module (gnu packages python)
+ #:use-module (gnu packages python-crypto)
#:use-module (gnu packages python-web)
#:use-module (gnu packages python-xyz)
#:use-module (gnu packages pulseaudio)
@@ -82,6 +91,7 @@
#:use-module (gnu packages sdl)
#:use-module (gnu packages sphinx)
#:use-module (gnu packages spice)
+ #:use-module (gnu packages ssh)
#:use-module (gnu packages texinfo)
#:use-module (gnu packages textutils)
#:use-module (gnu packages tls)
@@ -349,6 +359,374 @@ server and embedded PowerPC, and S390 guests.")
"usbredir" "libdrm" "libepoxy" "pulseaudio" "vde2"
"libcacard")))))
+(define (system->qemu-target system)
+ (cond
+ ((string-prefix? "i686" system)
+ "qemu-system-i386")
+ ((string-prefix? "arm" system)
+ "qemu-system-arm")
+ (else
+ (string-append "qemu-system-" (match (string-split system #\-)
+ ((arch kernel) arch)
+ (_ system))))))
+
+(define-public ganeti
+ (package
+ (name "ganeti")
+ (version "3.0.0.beta1")
+ (source (origin
+ (method url-fetch)
+ (uri (string-append "https://github.com/ganeti/ganeti/releases"
+ "/download/v3.0.0beta1/ganeti-"
+ version ".tar.gz"))
+ (sha256
+ (base32 "194mddbbcp8kvqhjxpx3sgvrzhlwfpdwk7m7brvdcrifrblbcd92"))
+ (patches (search-patches "ganeti-shepherd-support.patch"
+ "ganeti-shepherd-master-failover.patch"
+ "ganeti-copy-hmac.patch"
+ "ganeti-openvswitch-may-exist.patch"
+ "ganeti-drbd-compat.patch"
+ "ganeti-haskell-pythondir.patch"
+ "ganeti-disable-version-symlinks.patch"
+ "ganeti-preserve-PYTHONPATH.patch"))))
+ (build-system gnu-build-system)
+ (arguments
+ `(#:imported-modules ((guix build haskell-build-system)
+ (guix build python-build-system)
+ ,@%gnu-build-system-modules)
+ #:modules ((ice-9 rdelim)
+ ((guix build haskell-build-system) #:prefix haskell:)
+ ((guix build python-build-system) #:select (python-version))
+ ,@%gnu-build-system-modules)
+
+ ;; The default test target includes a lot of checks that are only really
+ ;; relevant for developers such as NEWS file checking, line lengths, etc.
+ ;; We are only interested in the "py-tests" and "hs-tests" targets: this
+ ;; is the closest we've got even though it includes a little more.
+ #:test-target "check-TESTS"
+
+ #:configure-flags
+ (list "--localstatedir=/var"
+ "--sharedstatedir=/var"
+ "--sysconfdir=/etc"
+ "--enable-haskell-tests"
+
+ ;; By default, the build system installs everything to versioned
+ ;; directories such as $libdir/3.0 and relies on a $libdir/default
+ ;; symlink pointed from /etc/ganeti/{lib,share} to actually function.
+ ;; This is done to accommodate installing multiple versions in
+ ;; parallel, but is of little use to us as Guix users can just
+ ;; roll back and forth. Thus, disable it for simplicity.
+ "--disable-version-links"
+
+ ;; Ganeti can optionally take control over SSH keys and rotate
+ ;; them with 'gnt-cluster renew-crypto', and thus needs to know
+ ;; how to restart the SSH server.
+ "--with-sshd-restart-command='herd restart ssh-daemon'"
+
+ ;; Look for OS definitions in this directory by default. It can
+ ;; be changed in the cluster configuration.
+ "--with-os-search-path=/run/current-system/profile/share/ganeti/os"
+
+ ;; The default QEMU executable to use. We don't use the package
+ ;; here because this entry is stored in the cluster configuration.
+ (string-append "--with-kvm-path=/run/current-system/profile/bin/"
+ ,(system->qemu-target (%current-system))))
+ #:phases
+ (modify-phases %standard-phases
+ (add-before 'bootstrap 'force-bootstrap
+ (lambda _
+ (delete-file "configure")
+ #t))
+ (add-after 'unpack 'adjust-quickcheck-requirement
+ (lambda _
+ ;; The test suite wants QuickCheck 2.12 or lower, but works fine
+ ;; with 2.13. Just bump the requirement.
+ (substitute* '("ganeti.cabal" "cabal/ganeti.template.cabal")
+ (("&& < 2\\.13")
+ "&& < 2.14"))
+ #t))
+ (add-after 'unpack 'patch-absolute-file-names
+ (lambda _
+ (substitute* '("lib/utils/process.py"
+ "lib/utils/text.py"
+ "src/Ganeti/Constants.hs"
+ "src/Ganeti/HTools/CLI.hs"
+ "test/py/ganeti.config_unittest.py"
+ "test/py/ganeti.hooks_unittest.py"
+ "test/py/ganeti.utils.process_unittest.py"
+ "test/py/ganeti.utils.text_unittest.py"
+ "test/py/ganeti.utils.wrapper_unittest.py")
+ (("/bin/sh") (which "sh"))
+ (("/bin/bash") (which "bash"))
+ (("/usr/bin/env") (which "env"))
+ (("/bin/true") (which "true")))
+
+ ;; This script is called by the node daemon at startup to perform
+ ;; sanity checks on the cluster IP addresses, and it is also used
+ ;; in a master-failover scenario. Add absolute references to
+ ;; avoid propagating these executables.
+ (substitute* "tools/master-ip-setup"
+ (("arping") (which "arping"))
+ (("ndisc6") (which "ndisc6"))
+ (("fping") (which "fping"))
+ (("grep") (which "grep"))
+ (("ip addr") (string-append (which "ip") " addr")))
+ #t))
+ (add-after 'unpack 'override-builtin-PATH
+ (lambda _
+ ;; Ganeti runs OS install scripts and similar with a built-in
+ ;; hard coded PATH. Patch so it works on Guix System.
+ (substitute* "src/Ganeti/Constants.hs"
+ (("/sbin:/bin:/usr/sbin:/usr/bin")
+ "/run/setuid-programs:/run/current-system/profile/sbin:\
+/run/current-system/profile/bin"))
+ #t))
+
+ ;; The build system invokes Cabal and GHC, which do not work with
+ ;; GHC_PACKAGE_PATH: <https://github.com/haskell/cabal/issues/3728>.
+ ;; Tweak the build system to do roughly what haskell-build-system does.
+ (add-before 'configure 'configure-haskell
+ (assoc-ref haskell:%standard-phases 'setup-compiler))
+ (add-after 'configure 'do-not-use-GHC_PACKAGE_PATH
+ (lambda _
+ (unsetenv "GHC_PACKAGE_PATH")
+ (substitute* "Makefile"
+ (("\\$\\(CABAL\\)")
+ "$(CABAL) --package-db=../package.conf.d")
+ (("\\$\\(GHC\\)")
+ "$(GHC) -package-db=../package.conf.d"))
+ #t))
+
+ (add-after 'configure 'fix-installation-directories
+ (lambda _
+ (substitute* "Makefile"
+ ;; Do not attempt to create /var during install.
+ (("\\$\\(DESTDIR\\)\\$\\{localstatedir\\}")
+ "$(DESTDIR)${prefix}${localstatedir}")
+ ;; Similarly, do not attempt to install the sample ifup scripts
+ ;; to /etc/ganeti.
+ (("\\$\\(DESTDIR\\)\\$\\(ifupdir\\)")
+ "$(DESTDIR)${prefix}$(ifupdir)"))
+ #t))
+ (add-before 'build 'adjust-tests
+ (lambda _
+ ;; Disable tests that can not run. Do it early to prevent
+ ;; touching the Makefile later and triggering a needless rebuild.
+ (substitute* "Makefile"
+ ;; These tests expect the presence of a 'root' user (via
+ ;; ganeti/runtime.py), which fails in the build environment.
+ (("test/py/ganeti\\.asyncnotifier_unittest\\.py") "")
+ (("test/py/ganeti\\.backend_unittest\\.py") "")
+ (("test/py/ganeti\\.daemon_unittest\\.py") "")
+ (("test/py/ganeti\\.tools\\.ensure_dirs_unittest\\.py") "")
+ (("test/py/ganeti\\.utils\\.io_unittest-runasroot\\.py") "")
+ ;; Disable the bash_completion test, as it requires the full
+ ;; bash instead of bash-minimal.
+ (("test/py/bash_completion\\.bash")
+ "")
+ ;; This test requires networking.
+ (("test/py/import-export_unittest\\.bash")
+ ""))
+
+ ;; Many of the Makefile targets reset PYTHONPATH before running
+ ;; the Python interpreter, which does not work very well for us.
+ (substitute* "Makefile"
+ (("PYTHONPATH=")
+ (string-append "PYTHONPATH=" (getenv "PYTHONPATH") ":")))
+ #t))
+ (add-after 'build 'build-bash-completions
+ (lambda _
+ (let ((orig-pythonpath (getenv "PYTHONPATH")))
+ (setenv "PYTHONPATH" (string-append ".:" orig-pythonpath))
+ (invoke "./autotools/build-bash-completion")
+ (setenv "PYTHONPATH" orig-pythonpath)
+ #t)))
+ (add-before 'check 'pre-check
+ (lambda* (#:key inputs #:allow-other-keys)
+ ;; Set TZDIR so that time zones are found.
+ (setenv "TZDIR" (string-append (assoc-ref inputs "tzdata")
+ "/share/zoneinfo"))
+
+ ;; This test checks whether PYTHONPATH is untouched, and extends
+ ;; it to include test directories if so. Add an else branch for
+ ;; our modified PYTHONPATH, in order to prevent a confusing test
+ ;; failure where expired certificates are not cleaned because
+ ;; check-cert-expired is silently crashing.
+ (substitute* "test/py/ganeti-cleaner_unittest.bash"
+ (("then export PYTHONPATH=(.*)" all testpath)
+ (string-append all "else export PYTHONPATH="
+ (getenv "PYTHONPATH") ":" testpath "\n")))
+
+ (substitute* "test/py/ganeti.utils.process_unittest.py"
+ ;; This test attempts to run an executable with
+ ;; RunCmd(..., reset_env=True), which fails because the default
+ ;; PATH from Constants.hs does not exist in the build container.
+ ((".*def testResetEnv.*" all)
+ (string-append " @unittest.skipIf(True, "
+ "\"cannot reset env in the build container\")\n"
+ all))
+
+ ;; XXX: Somehow this test fails in the build container, but
+ ;; works in 'guix environment -C', even without /bin/sh?
+ ((".*def testPidFile.*" all)
+ (string-append " @unittest.skipIf(True, "
+ "\"testPidFile fails in the build container\")\n"
+ all)))
+
+ ;; XXX: Why are these links not added automatically.
+ (with-directory-excursion "test/hs"
+ (for-each (lambda (file)
+ (symlink "../../src/htools" file))
+ '("hspace" "hscan" "hinfo" "hbal" "hroller"
+ "hcheck" "hail" "hsqueeze")))
+ #t))
+ (add-after 'install 'install-bash-completions
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let* ((out (assoc-ref outputs "out"))
+ (compdir (string-append out "/etc/bash_completion.d")))
+ (mkdir-p compdir)
+ (copy-file "doc/examples/bash_completion"
+ (string-append compdir "/ganeti"))
+ ;; The one file contains completions for many different
+ ;; executables. Create symlinks for found completions.
+ (with-directory-excursion compdir
+ (for-each
+ (lambda (prog) (symlink "ganeti" prog))
+ (call-with-input-file "ganeti"
+ (lambda (port)
+ (let loop ((line (read-line port))
+ (progs '()))
+ (if (eof-object? line)
+ progs
+ (if (string-prefix? "complete" line)
+ (loop (read-line port)
+ ;; Extract "prog" from lines of the form:
+ ;; "complete -F _prog -o filenames prog".
+ ;; Note that 'burnin' is listed with the
+ ;; absolute file name, which is why we
+ ;; run everything through 'basename'.
+ (cons (basename (car (reverse (string-split
+ line #\ ))))
+ progs))
+ (loop (read-line port) progs))))))))
+ #t)))
+ ;; Wrap all executables with PYTHONPATH. We can't borrow the phase
+ ;; from python-build-system because we also need to wrap the scripts
+ ;; in $out/lib/ganeti such as "node-daemon-setup".
+ (add-after 'install 'wrap
+ (lambda* (#:key inputs outputs #:allow-other-keys)
+ (let* ((out (assoc-ref outputs "out"))
+ (sbin (string-append out "/sbin"))
+ (lib (string-append out "/lib"))
+ (python (assoc-ref inputs "python"))
+ (major+minor (python-version python))
+ (PYTHONPATH (string-append lib "/python" major+minor
+ "/site-packages:"
+ (getenv "PYTHONPATH"))))
+ (define (shell-script? file)
+ (call-with-ascii-input-file file
+ (lambda (port)
+ (let ((shebang (false-if-exception (read-line port))))
+ (and shebang
+ (string-prefix? "#!" shebang)
+ (or (string-contains shebang "/bin/bash")
+ (string-contains shebang "/bin/sh")))))))
+
+ (define (wrap? file)
+ ;; Do not wrap shell scripts because some are meant to be
+ ;; sourced, which breaks if they are wrapped. We do wrap
+ ;; the Haskell executables because some call out to Python
+ ;; directly.
+ (and (executable-file? file)
+ (not (symbolic-link? file))
+ (not (shell-script? file))))
+
+ (for-each (lambda (file)
+ (wrap-program file
+ `("PYTHONPATH" ":" prefix (,PYTHONPATH))))
+ (filter wrap?
+ (append (find-files (string-append lib "/ganeti"))
+ (find-files sbin))))
+ #t))))))
+ (native-inputs
+ `(("haskell" ,ghc)
+ ("cabal" ,cabal-install)
+ ("m4" ,m4)
+
+ ;; These inputs are necessary to bootstrap the package, because we
+ ;; have patched the build system.
+ ("autoconf" ,autoconf)
+ ("automake" ,automake)
+
+ ;; Test dependencies.
+ ("fakeroot" ,fakeroot)
+ ("ghc-temporary" ,ghc-temporary)
+ ("ghc-test-framework" ,ghc-test-framework)
+ ("ghc-test-framework-hunit" ,ghc-test-framework-hunit)
+ ("ghc-test-framework-quickcheck2" ,ghc-test-framework-quickcheck2)
+ ("python-mock" ,python-mock)
+ ("python-pyyaml" ,python-pyyaml)
+ ("openssh" ,openssh)
+ ("procps" ,procps)
+ ("shelltestrunner" ,shelltestrunner)
+ ("tzdata" ,tzdata-for-tests)))
+ (inputs
+ `(("arping" ,iputils) ;must be the iputils version
+ ("curl" ,curl)
+ ("fping" ,fping)
+ ("iproute2" ,iproute)
+ ("ndisc6" ,ndisc6)
+ ("socat" ,socat)
+ ("qemu" ,qemu-minimal) ;for qemu-img
+ ("ghc-attoparsec" ,ghc-attoparsec)
+ ("ghc-base64-bytestring" ,ghc-base64-bytestring)
+ ("ghc-cryptonite" ,ghc-cryptonite)
+ ("ghc-curl" ,ghc-curl)
+ ("ghc-hinotify" ,ghc-hinotify)
+ ("ghc-hslogger" ,ghc-hslogger)
+ ("ghc-json" ,ghc-json)
+ ("ghc-lens" ,ghc-lens)
+ ("ghc-lifted-base" ,ghc-lifted-base)
+ ("ghc-network" ,ghc-network)
+ ("ghc-old-time" ,ghc-old-time)
+ ("ghc-psqueue" ,ghc-psqueue)
+ ("ghc-regex-pcre" ,ghc-regex-pcre)
+ ("ghc-utf8-string" ,ghc-utf8-string)
+ ("ghc-zlib" ,ghc-zlib)
+
+ ;; For the optional metadata daemon.
+ ("ghc-snap-core" ,ghc-snap-core)
+ ("ghc-snap-server" ,ghc-snap-server)
+
+ ("python" ,python)
+ ("python-pyopenssl" ,python-pyopenssl)
+ ("python-simplejson" ,python-simplejson)
+ ("python-pyparsing" ,python-pyparsing)
+ ("python-pyinotify" ,python-pyinotify)
+ ("python-pycurl" ,python-pycurl)
+ ("python-bitarray" ,python-bitarray)
+ ("python-paramiko" ,python-paramiko)
+ ("python-psutil" ,python-psutil)))
+ (home-page "http://www.ganeti.org/")
+ (synopsis "Cluster-based virtual machine management system")
+ (description
+ "Ganeti is a virtual machine management tool built on top of existing
+virtualization technologies such as Xen or KVM. Once installed, Ganeti
+assumes management of the virtual instances (Xen DomU). Ganeti controls:
+
+@itemize @bullet
+@item Disk creation management;
+@item Operating system installation for instances (in co-operation with
+OS-specific install scripts); and
+@item Startup, shutdown, and failover between physical systems.
+@end itemize
+
+Ganeti is designed to facilitate cluster management of virtual servers and
+to provide fast and simple recovery after physical failures, using
+commodity hardware.")
+ (license license:bsd-2)))
+
(define-public libosinfo
(package
(name "libosinfo")
--
2.27.0
guix-patches <at> gnu.org:bug#42261; Package guix-patches.
(Wed, 08 Jul 2020 10:13:02 GMT) Full text and rfc822 format available.Message #17 received at 42261 <at> debbugs.gnu.org (full text, mbox):
From: Marius Bakke <marius <at> gnu.org> To: 42261 <at> debbugs.gnu.org Subject: [PATCH 4/4] services: Add ganeti. Date: Wed, 8 Jul 2020 12:11:17 +0200
* gnu/services/virtualization.scm (<ganeti-noded-configuration>,
<ganeti-confd-configuration>, <ganeti-wconfd-configuration>,
<ganeti-luxid-configuration>, <ganeti-rapi-configuration>,
<ganeti-kvmd-configuration>, <ganeti-mond-configuration>),
<ganeti-metad-configuration>, <ganeti-watcher-configuration>,
<ganeti-cleaner-configuration>, <ganeti-configuration>, <ganeti-os>,
<ganeti-os-variant>, <debootstrap-configuration>): New record types.
(%default-ganeti-environment-variables, ganeti-noded-service,
ganeti-noded-service-type, ganeti-confd-service, ganeti-confd-service-type,
ganeti-wconfd-service, ganeti-wconfd-service-type, ganeti-luxid-service,
ganeti-luxid-service-type, ganeti-rapi-service, ganeti-rapi-service-type,
ganeti-kvmd-service, ganeti-kvmd-service-type, ganeti-mond-service,
ganeti-mond-service-type, ganeti-metad-service, ganeti-metad-service-type,
ganeti-watcher-command, ganeti-watcher-jobs, ganeti-watcher-service-type,
ganeti-cleaner-jobs, ganeti-cleaner-service-type, ganeti-activation,
ganeti-shepherd-services, ganeti-mcron-jobs, ganeti-service-type,
hooks->directory, debootstrap-configuration-compiler, debootstrap-variant,
ganeti-os-variant->configuration, ganeti-os->directory, ganeti-directory,
file-storage-file, ganeti-etc-service, ganeti-service-type): New variables.
* gnu/tests/virtualization.scm (%debootstrap-hooks, %ganeti-os,
run-ganeti-test, %test-ganeti-kvm, %test-ganeti-lxc): New variables.
* doc/guix.texi (Virtualization Services): Document accordingly.
---
doc/guix.texi | 560 ++++++++++++++++++++
gnu/services/virtualization.scm | 906 +++++++++++++++++++++++++++++++-
gnu/tests/virtualization.scm | 175 +++++-
3 files changed, 1639 insertions(+), 2 deletions(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index 992bc303bb..d5cfc23297 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -24898,6 +24898,566 @@ the @code{--snapshot} flag using something along these lines:
(options '("--hda"))))
@end lisp
+@subsubheading Ganeti
+
+@cindex ganeti
+
+@quotation Note
+This service currently offered as a tech preview. Configuration options may
+be changed in a backwards-incompatible manner, and not all features have been
+thorougly tested. Users of this service are encouraged to share their
+experience at @email{guix-devel@@gnu.org}.
+@end quotation
+
+Ganeti is a cluster-based virtual machine management system. It consists
+of multiple services which are described later in this section. In addition
+to the Ganeti service, you will need the OpenSSH service
+(@pxref{Networking Services, @code{openssh-service-type}}), and update the
+@file{/etc/hosts} file (@pxref{operating-system Reference, @code{hosts-file}})
+with the cluster name and address (or use a DNS server). Here is an example
+configuration for a Ganeti cluster node:
+
+@lisp
+(operating-system
+ ...
+ (host-name "node1")
+ (hosts-file (plain-file "hosts" (format #f "
+127.0.0.1 localhost
+::1 localhost
+
+192.168.1.100 ganeti.example.com
+192.168.1.101 node1.example.com node1
+192.168.1.102 node2.example.com node2
+")))
+
+ ;; Install QEMU so we can use KVM-based instances, and LVM, DRBD and Ceph
+ ;; in order to use the "plain", "drbd" and "rbd" storage backends.
+ (packages (append (map specification->package
+ '("qemu" "lvm2" "drbd-utils" "ceph"
+ "ganeti-instance-guix" "ganeti-instance-debootstrap"))
+ %base-packages))
+ (services
+ (append (list (static-networking-service "eth0" "192.168.1.101"
+ #:netmask "255.255.255.0"
+ #:gateway "192.168.1.254"
+ #:name-servers '("192.168.1.252"
+ "192.168.1.253"))
+
+ ;; Ganeti uses SSH to communicate between nodes.
+ (service openssh-service-type
+ (openssh-configuration
+ (permit-root-login 'without-password)))
+
+ (service ganeti-service-type
+ (ganeti-configuration
+ ;; This file specifies allowed file system paths
+ ;; for storing virtual machine images.
+ (file-storage-paths '("/srv/ganeti/file-storage")))))
+ %base-services)))
+@end lisp
+
+Users are advised to read the
+@url{http://docs.ganeti.org/ganeti/master/html/admin.html,Ganeti
+administrators guide} to learn about the various cluster options and
+day-to-day operations.
+
+There is also a
+@url{https://guix.gnu.org/blog/2020/ganeti-cluster-on-guix/,blog post}
+describing how to configure a small cluster.
+
+The rest of this section documents each of the various services and their
+configuration options.
+
+@defvr {Scheme Variable} ganeti-service-type
+This is a service type that includes all the various services that Ganeti
+nodes should run.
+
+Its value is a @code{ganeti-configuration} object that defines the package
+to use for CLI operations, as well as configuration for the various daemons.
+
+@end defvr
+
+@deftp {Data Type} ganeti-configuration
+The @code{ganeti} service takes the following configuration options:
+
+@table @asis
+@item @code{ganeti} (default: @code{ganeti})
+The @code{ganeti} package to use. It will be installed to the system profile
+and make @command{gnt-cluster}, @command{gnt-instance}, etc available. Note
+that the value specified here does not affect the other services as each refer
+to a specific @code{ganeti} package (see below).
+
+@item @code{noded-configuration} (default: @code{(ganeti-noded-configuration)})
+@item @code{confd-configuration} (default: @code{(ganeti-confd-configuration)})
+@item @code{wconfd-configuration} (default: @code{(ganeti-wconfd-configuration)})
+@item @code{luxid-configuration} (default: @code{(ganeti-luxid-configuration)})
+@item @code{rapi-configuration} (default: @code{(ganeti-rapi-configuration)})
+@item @code{kvmd-configuration} (default: @code{(ganeti-kvmd-configuration)})
+@item @code{mond-configuration} (default: @code{(ganeti-mond-configuration)})
+@item @code{watcher-configuration} (default: @code{(ganeti-watcher-configuration)})
+@item @code{cleaner-configuration} (default: @code{(ganeti-cleaner-configuration)})
+
+These options control the various daemons and cron jobs that are distributed
+with Ganeti. The possible values for these are described in detail below.
+To override a setting, you must use the configuration type for that service:
+
+@lisp
+(service ganeti-service-type
+ (ganeti-configuration
+ (rapi-configuration
+ (ganeti-rapi-configuration
+ (interface "eth1"))))
+ (watcher-configuration
+ (ganeti-watcher-configuration
+ (rapi-ip "10.0.0.1"))))
+@end lisp
+
+@item @code{file-storage-paths} (default: @code{'()})
+List of allowed directories for file storage backend.
+
+@item @code{os} (default: @code{'()})
+List if @code{<ganeti-os>} records. This currently only works with the
+@code{debootstrap} OS plugin.
+@end table
+
+In essence @code{ganeti-service-type} is shorthand for declaring each service
+individually:
+
+@lisp
+(service ganeti-noded-service-type)
+(service ganeti-confd-service-type)
+(service ganeti-wconfd-service-type)
+(service ganeti-luxid-service-type)
+(service ganeti-kvmd-service-type)
+(service ganeti-mond-service-type)
+(service ganeti-watcher-service-type)
+(service ganeti-cleaner-service-type)
+@end lisp
+
+The @file{/etc/ganeti} directory would need to be managed by other means however.
+
+@end deftp
+
+@deftp {Scheme Variable} ganeti-os
+This data type is suitable for passing to the @code{os} configuration of
+Ganeti. It takes the following parameters:
+
+@table @asis
+@item @code{name}
+The name for this OS. It is only used to specify where the variant configuration
+ends up. Setting it to "debootstrap" will create
+@file{/etc/ganeti/instance-debootstrap}.
+
+@item @code{variants} (default: @code{'()})
+List of @code{ganeti-os-variant} objects for this OS.
+
+@end table
+@end deftp
+
+@deftp {Scheme Variable} ganeti-os-variant
+This is the data type for a Ganeti OS variant.
+
+@table @asis
+@item @code{name}
+The name of this variant.
+
+@item @code{configuration}
+A configuration for this variant. Currently only @code{debootstrap-configuration}
+is supported.
+
+@end table
+@end deftp
+
+@deftp {Scheme Variable} debootstrap-configuration
+
+@table @asis
+@item @code{hooks} (default: @code{#f})
+If set, this must be a G-expression that specifies a directory with scripts
+that will run when the OS is installed. It can also be a list of
+@code{(name . file-like)} pairs. For example:
+
+@lisp
+
+`((10-testing . ,(plain-file "#!/bin/sh\necho Hello, World")))
+
+@end lisp
+
+That will create a directory with one executable and run it every time this
+variant is installed.
+@item @code{proxy} (default: @code{#f})
+HTTP proxy to use, if any.
+@item @code{mirror} (default: @var{#f})
+The Debian mirror. Typically something like
+@code{http://ftp.no.debian.org/debian}. The default varies depending on
+the distribution.
+@item @code{arch} (default: @code{#f})
+The dpkg architecture. Set to @code{armhf} to debootstrap an ARMv7 instance
+on an AArch64 host. Default is to use the current system architecture.
+@item @code{suite} (default: @code{"stable"})
+When not #f, must be a Debian distribution suite such as @code{buster} or
+@code{focal}.
+@item @code{extra-pkgs} (default: @var{%default-debootstrap-extra-pkgs})
+List of extra packages that will get installed by dpkg in addition
+to the minimal system.
+@item @code{components} (default: @code{#f})
+When set, must be a list of Debian repository ``components''. For example
+@code{'("main" "contrib")}.
+@item @code{generate-cache?} (default: @code{#t})
+Whether to automatically cache the generated debootstrap archive.
+@item @code{clean-cache} (default: @code{14})
+Discard the cache after this amount of days. Use @code{#f} to never
+clear the cache.
+@item @code{partition-style} (default: @code{'msdos})
+The type of partition to create. When set, it must be one of
+@code{'msdos}, @code{'none} or a string.
+@item @code{partition-alignment} (default: @code{2048})
+Alignment of the partition in sectors.
+@end table
+@end deftp
+
+@defvr {Scheme Variable} debootstrap-variant
+This is a helper procedure to create @code{ganeti-os-variant} records. It
+takes two arguments: a name and an optional @code{debootstrap-configuration}
+for this variant.
+@end defvr
+
+@defvr {Scheme Variable} ganeti-noded-service-type
+@command{ganeti-noded} is the daemon responsible for node-specific functions
+within the Ganeti system. The value of this service must be a
+@code{ganeti-noded-configuration} object.
+
+@end defvr
+
+@deftp {Data Type} ganeti-noded-configuration
+This is the configuration for the @code{ganeti-noded} service.
+
+@table @asis
+@item @code{ganeti} (default: @code{ganeti})
+The @code{ganeti} package to use for this service.
+
+@item @code{port} (default: @code{1811})
+The TCP port on which the node daemon listens for network requests.
+
+@item @code{address} (default: @code{"0.0.0.0"})
+The network address that the daemon will bind to. The default address means
+bind to all available addresses.
+
+@item @code{interface} (default: @code{#f})
+When this is set, it must be a specific network interface (e.g.@: @code{eth0})
+that the daemon will bind to.
+
+@item @code{max-clients} (default: @code{20})
+This sets a limit on the maximum number of simultaneous client connections
+that the daemon will handle. Connections above this count are accepted, but
+no responses will be sent until enough connections have closed.
+
+@item @code{ssl?} (default: @code{#t})
+Whether to use SSL/TLS to encrypt network communications. The certificate
+is automatically provisioned by the cluster and can be rotated with
+@command{gnt-cluster renew-crypto}.
+
+@item @code{ssl-key} (default: @file{"/var/lib/ganeti/server.pem"})
+This can be used to provide a specific encryption key for TLS communications.
+
+@item @code{ssl-cert} (default: @file{"/var/lib/ganeti/server.pem"})
+This can be used to provide a specific certificate for TLS communications.
+
+@item @code{debug?} (default: @code{#f})
+When true, the daemon performs additional logging for debugging purposes.
+Note that this @emph{will} leak encryption details to the log files, use
+with care.
+
+@end table
+@end deftp
+
+@defvr {Scheme Variable} ganeti-confd-service-type
+@command{ganeti-confd} answers queries related to the configuration of a
+Ganeti cluster. The purpose of this daemon is to have a highly available
+and fast way to query cluster configuration values. It is automatically
+active on all @dfn{master candidates}. The value of this service must be a
+@code{ganeti-confd-configuration} object.
+
+@end defvr
+
+@deftp {Data Type} ganeti-confd-configuration
+This is the configuration for the @code{ganeti-confd} service.
+
+@table @asis
+@item @code{ganeti} (default: @code{ganeti})
+The @code{ganeti} package to use for this service.
+
+@item @code{port} (default: @code{1814})
+The UDP port on which to listen for network requests.
+
+@item @code{address} (default: @code{"0.0.0.0"})
+Network address that the daemon will bind to.
+
+@item @code{debug?} (default: @code{#f})
+When true, the daemon performs additional logging for debugging purposes.
+
+@end table
+@end deftp
+
+@defvr {Scheme Variable} ganeti-wconfd-service-type
+@command{ganeti-wconfd} is the daemon that has authoritative knowledge
+about the cluster configuration and is the only entity that can accept
+changes to it. All jobs that need to modify the configuration will do so
+by sending appropriate requests to this daemon. It only runs on the
+@dfn{master node} and will automatically disable itself on other nodes.
+
+The value of this service must be a
+@code{ganeti-wconfd-configuration} object.
+@end defvr
+
+@deftp {Data Type} ganeti-wconfd-configuration
+This is the configuration for the @code{ganeti-wconfd} service.
+
+@table @asis
+@item @code{ganeti} (default: @code{ganeti})
+The @code{ganeti} package to use for this service.
+
+@item @code{no-voting?} (default: @code{#f})
+The daemon will refuse to start if the majority of cluster nodes does not
+agree that it is running on the master node. Set to @code{#t} to start
+even if a quorum can not be reached (dangerous, use with caution).
+
+@item @code{debug?} (default: @code{#f})
+When true, the daemon performs additional logging for debugging purposes.
+
+@end table
+@end deftp
+
+@defvr {Scheme Variable} ganeti-luxid-service-type
+@command{ganeti-luxid} is a daemon used to answer queries related to the
+configuration and the current live state of a Ganeti cluster. Additionally,
+it is the authorative daemon for the Ganeti job queue. Jobs can be
+submitted via this daemon and it schedules and starts them.
+
+It takes a @code{ganeti-luxid-configuration} object.
+@end defvr
+
+@deftp {Data Type} ganeti-luxid-configuration
+This is the configuration for the @code{ganeti-wconfd} service.
+
+@table @asis
+@item @code{ganeti} (default: @code{ganeti})
+The @code{ganeti} package to use for this service.
+
+@item @code{no-voting?} (default: @code{#f})
+The daemon will refuse to start if it cannot verify that the majority of
+cluster nodes believes that it is running on the master node. Set to
+@code{#t} to ignore such checks and start anyway (this can be dangerous).
+
+@item @code{debug?} (default: @code{#f})
+When true, the daemon performs additional logging for debugging purposes.
+
+@end table
+@end deftp
+
+@defvr {Scheme Variable} ganeti-rapi-service-type
+@command{ganeti-rapi} provides a remote API for Ganeti clusters. It runs on
+the master node and can be used to perform cluster actions programmatically
+via a JSON-based RPC protocol.
+
+Most query operations are allowed without authentication (unless
+@var{require-authentication?} is set), whereas write operations require
+explicit authorization via the @file{/var/lib/ganeti/rapi/users} file. See
+the @url{http://docs.ganeti.org/ganeti/master/html/rapi.html, Ganeti Remote
+API documentation} for more information.
+
+The value of this service must be a @code{ganeti-rapi-configuration} object.
+@end defvr
+
+@deftp {Data Type} ganeti-rapi-configuration
+This is the configuration for the @code{ganeti-rapi} service.
+
+@table @asis
+@item @code{ganeti} (default: @code{ganeti})
+The @code{ganeti} package to use for this service.
+
+@item @code{require-authentication?} (default: @code{#f})
+Whether to require authentication even for read-only operations.
+
+@item @code{port} (default: @code{5080})
+The TCP port on which to listen to API requests.
+
+@item @code{address} (default: @code{"0.0.0.0"})
+The network address that the service will bind to. By default it listens
+on all configured addresses.
+
+@item @code{interface} (default: @code{#f})
+When set, it must specify a specific network interface such as @code{eth0}
+that the daemon will bind to.
+
+@item @code{max-clients} (default: @code{20})
+The maximum number of simultaneous client requests to handle. Further
+connections are allowed, but no responses are sent until enough connections
+have closed.
+
+@item @code{ssl-key} (default: @file{"/var/lib/ganeti/server.pem"})
+This can be used to provide a specific encryption key for TLS communications.
+
+@item @code{ssl-cert} (default: @file{"/var/lib/ganeti/server.pem"})
+This can be used to provide a specific certificate for TLS communications.
+
+@item @code{debug?} (default: @code{#f})
+When true, the daemon performs additional logging for debugging purposes.
+Note that this @emph{will} leak encryption details to the log files, use
+with caution.
+
+@end table
+@end deftp
+
+@defvr {Scheme Variable} ganeti-kvmd-service-type
+@command{ganeti-kvmd} is responsible for determining whether a given KVM
+instance was shut down by an administrator or a user. Normally Ganeti will
+restart an instance that was not stopped through Ganeti itself. If the
+cluster option @code{user_shutdown} is true, this daemon monitors the
+@code{QMP} socket provided by QEMU and listens for shutdown events, and
+marks the instance as @dfn{USER_down} instead of @dfn{ERROR_down} when
+it shuts down gracefully by itself.
+
+It takes a @code{ganeti-kvmd-configuration} object.
+@end defvr
+
+@deftp {Data Type} ganeti-kvmd-configuration
+
+@table @asis
+@item @code{ganeti} (default: @code{ganeti})
+The @code{ganeti} package to use for this service.
+
+@item @code{debug?} (default: @code{#f})
+When true, the daemon performs additional logging for debugging purposes.
+
+@end table
+@end deftp
+
+@defvr {Scheme Variable} ganeti-mond-service-type
+@command{ganeti-mond} is an optional daemon that provides Ganeti monitoring
+functionality. It is responsible for running data collectors and publish the
+collected information through a HTTP interface.
+
+It takes a @code{ganeti-mond-configuration} object.
+@end defvr
+
+@deftp {Data Type} ganeti-mond-configuration
+
+@table @asis
+@item @code{ganeti} (default: @code{ganeti})
+The @code{ganeti} package to use for this service.
+
+@item @code{port} (default: @code{1815})
+The port on which the daemon will listen.
+
+@item @code{address} (default: @code{"0.0.0.0"})
+The network address that the daemon will bind to. By default it binds to all
+available interfaces.
+
+@item @code{debug?} (default: @code{#f})
+When true, the daemon performs additional logging for debugging purposes.
+
+@end table
+@end deftp
+
+@defvr {Scheme Variable} ganeti-metad-service-type
+@command{ganeti-metad} is an optional daemon that can be used to provide
+information about the cluster to instances or OS install scripts. It is
+not included in @code{ganeti-service-type} because using it requires
+additional configuration and support in OS providers.
+
+It takes a @code{ganeti-metad-configuration} object.
+@end defvr
+
+@deftp {Data Type} ganeti-metad-configuration
+
+@table @asis
+@item @code{ganeti} (default: @code{ganeti})
+The @code{ganeti} package to use for this service.
+
+@item @code{port} (default: @code{80})
+The port on which the daemon will listen.
+
+@item @code{address} (default: @code{#f})
+If set, the daemon will bind to this address only. If left unset, the behavior
+depends on the cluster configuration.
+
+@item @code{debug?} (default: @code{#f})
+When true, the daemon performs additional logging for debugging purposes.
+
+@end table
+@end deftp
+
+@defvr {Scheme Variable} ganeti-watcher-service-type
+@command{ganeti-watcher} is a script designed to run periodically and ensure
+the health of a cluster. It will automatically restart instances that have
+stopped without Ganetis consent, and repairs DRBD links in case a node has
+rebooted. It also archives old cluster jobs and restarts Ganeti daemons
+that are not running. If the cluster parameter @code{ensure_node_health}
+is set, the watcher will also shutdown instances and DRBD devices if the
+node it is running on is declared offline by known master candidates.
+
+It can be paused on all nodes with @command{gnt-cluster watcher pause}.
+
+The service takes a @code{ganeti-watcher-configuration} object.
+@end defvr
+
+@deftp {Data Type} ganeti-watcher-configuration
+
+@table @asis
+@item @code{ganeti} (default: @code{ganeti})
+The @code{ganeti} package to use for this service.
+
+@item @code{schedule} (default: @code{'(next-second-from (next-minute (range 0 60 5)))})
+How often to run the script. The default is every five minutes.
+
+@item @code{rapi-ip} (default: @code{#f})
+This option needs to be specified only if the RAPI daemon is configured to use
+a particular interface or address. By default the cluster address is used.
+
+@item @code{job-age} (default: @code{(* 6 3600)})
+Archive cluster jobs older than this age, specified in seconds. The default
+is 6 hours. This keeps @command{gnt-job list} manageable.
+
+@item @code{verify-disks?} (default: @code{#t})
+If this is @code{#f}, the watcher will not try to repair broken DRBD links
+automatically. Administrators should instead use
+@command{gnt-cluster verify-disks} manually.
+
+@item @code{debug?} (default: @code{#f})
+When @code{#t}, the script performs additional logging for debugging purposes.
+
+@end table
+@end deftp
+
+@defvr {Scheme Variable} ganeti-cleaner-service-type
+@command{ganeti-cleaner} is a script designed to run periodically and remove
+old files from the cluster. This service type controls two @dfn{cron jobs}:
+one intended for the master node that permanently purges old cluster jobs,
+and one intended for every node that removes expired X509 certificates, keys,
+and outdated @command{ganeti-watcher} information. Like all Ganeti services,
+it is safe to include even on non-master nodes as it will disable itself as
+necessary.
+
+It takes a @code{ganeti-cleaner-configuration} object.
+@end defvr
+
+@deftp {Data Type} ganeti-cleaner-configuration
+
+@table @asis
+@item @code{ganeti} (default: @code{ganeti})
+The @code{ganeti} package to use for the @command{gnt-cleaner} command.
+
+@item @code{master-schedule} (default: @code{"45 1 * * *"})
+How often to run the master cleaning job. The default is once per day, at
+01:45:00.
+
+@item @code{node-schedule} (default: @code{"45 2 * * *"})
+How often to run the node cleaning job. The default is once per day, at
+02:45:00.
+
+@end table
+@end deftp
+
@node Version Control Services
@subsection Version Control Services
diff --git a/gnu/services/virtualization.scm b/gnu/services/virtualization.scm
index b93ed70099..0566508a3a 100644
--- a/gnu/services/virtualization.scm
+++ b/gnu/services/virtualization.scm
@@ -2,6 +2,7 @@
;;; Copyright © 2017 Ryan Moe <ryan.moe <at> gmail.com>
;;; Copyright © 2018 Ludovic Courtès <ludo <at> gnu.org>
;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <janneke <at> gnu.org>
+;;; Copyright © 2020 Marius Bakke <marius <at> gnu.org>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -28,6 +29,7 @@
#:use-module (gnu services base)
#:use-module (gnu services configuration)
#:use-module (gnu services dbus)
+ #:use-module (gnu services mcron)
#:use-module (gnu services shepherd)
#:use-module (gnu services ssh)
#:use-module (gnu services)
@@ -45,10 +47,12 @@
#:use-module (guix store)
#:use-module (guix utils)
+ #:use-module (srfi srfi-1)
#:use-module (srfi srfi-9)
#:use-module (srfi srfi-26)
#:use-module (rnrs bytevectors)
#:use-module (ice-9 match)
+ #:use-module (ice-9 format)
#:export (%hurd-vm-operating-system
hurd-vm-configuration
@@ -77,7 +81,65 @@
qemu-binfmt-configuration
qemu-binfmt-configuration?
- qemu-binfmt-service-type))
+ qemu-binfmt-service-type
+
+ ganeti-noded-configuration
+ ganeti-noded-configuration?
+ ganeti-noded-service-type
+
+ ganeti-confd-configuration
+ ganeti-confd-configuration?
+ ganeti-confd-service-type
+
+ ganeti-wconfd-configuration
+ ganeti-wconfd-configuration?
+ ganeti-wconfd-service-type
+ ganeti-wconfd-forced-service-type
+
+ ganeti-luxid-configuration
+ ganeti-luxid-configuration?
+ ganeti-luxid-service-type
+
+ ganeti-rapi-configuration
+ ganeti-rapi-configuration?
+ ganeti-rapi-service-type
+
+ ganeti-kvmd-configuration
+ ganeti-kvmd-configuration?
+ ganeti-kvmd-service-type
+
+ ganeti-mond-configuration
+ ganeti-mond-configuration?
+ ganeti-mond-service-type
+
+ ganeti-metad-configuration
+ ganeti-metad-configuration?
+ ganeti-metad-service-type
+
+ ganeti-watcher-configuration
+ ganeti-watcher-configuration?
+ ganeti-watcher-service-type
+
+ ganeti-cleaner-configuration
+ ganeti-cleaner-configuration?
+ ganeti-cleaner-service-type
+
+ ganeti-os
+ ganeti-os?
+ ganeti-os-variants
+
+ ganeti-os-variant
+ ganeti-os-variant?
+ ganeti-os-variant-configuration
+
+ %default-debootstrap-extra-pkgs
+ debootstrap-configuration
+ debootstrap-configuration?
+ debootstrap-variant
+
+ ganeti-configuration
+ ganeti-configuration?
+ ganeti-service-type))
(define (uglify-field-name field-name)
(let ((str (symbol->string field-name)))
@@ -912,3 +974,845 @@ functionality of the kernel Linux.")))
(default-value (hurd-vm-configuration))
(description
"Provide a Virtual Machine running the GNU/Hurd.")))
+
+
+
+;;;
+;;; Service definitions for running a Ganeti cluster.
+;;;
+;;; Planned improvements: run daemons (except ganeti-noded) under unprivileged
+;;; user accounts and/or containers. The account names must match the ones
+;;; given to Ganetis configure script. metad needs "setcap" or root in order
+;;; to bind on port 80.
+
+;; Set PATH so the various daemons are able to find the 'ip' executable, LVM,
+;; Ceph, Gluster, etc, without having to add absolute references to everything.
+(define %default-ganeti-environment-variables
+ (list (string-append "PATH="
+ (string-join '("/run/setuid-programs"
+ "/run/current-system/profile/sbin"
+ "/run/current-system/profile/bin")
+ ":"))))
+
+(define-record-type* <ganeti-noded-configuration>
+ ganeti-noded-configuration make-ganeti-noded-configuration
+ ganeti-noded-configuration?
+ (ganeti ganeti-noded-configuration-ganeti ;<package>
+ (default ganeti))
+ (port ganeti-noded-configuration-port ;integer
+ (default 1811))
+ (address ganeti-noded-configuration-address ;string
+ (default "0.0.0.0"))
+ (interface ganeti-noded-configuration-interface ;string | #f
+ (default #f))
+ (max-clients ganeti-noded-configuration-max-clients ;integer
+ (default 20))
+ (ssl? ganeti-noded-configuration-ssl? ;Boolean
+ (default #t))
+ (ssl-key ganeti-noded-configuration-ssl-key ;string
+ (default "/var/lib/ganeti/server.pem"))
+ (ssl-cert ganeti-noded-configuration-ssl-cert ;string
+ (default "/var/lib/ganeti/server.pem"))
+ (debug? ganeti-noded-configuration-debug? ;Boolean
+ (default #f)))
+
+(define ganeti-noded-service
+ (match-lambda
+ (($ <ganeti-noded-configuration> ganeti port address interface max-clients
+ ssl? ssl-key ssl-cert debug?)
+ (list (shepherd-service
+ (documentation "Run the Ganeti node daemon.")
+ (provision '(ganeti-noded))
+ (requirement '(user-processes networking))
+
+ ;; If the daemon stops, it is probably for a good reason;
+ ;; otherwise ganeti-watcher will restart it for us anyway.
+ (respawn? #f)
+
+ (start #~(make-forkexec-constructor
+ (list #$(file-append ganeti "/sbin/ganeti-noded")
+ (string-append "--port=" (number->string #$port))
+ (string-append "--bind=" #$address)
+ #$@(if interface
+ #~((string-append "--interface=" #$interface))
+ #~())
+ (string-append "--max-clients="
+ #$(number->string max-clients))
+ #$@(if ssl?
+ #~((string-append "--ssl-key=" #$ssl-key)
+ (string-append "--ssl-cert=" #$ssl-cert))
+ #~("--no-ssl"))
+ #$@(if debug?
+ #~("--debug")
+ #~()))
+ #:environment-variables
+ '#$%default-ganeti-environment-variables
+ #:pid-file "/var/run/ganeti/ganeti-noded.pid"))
+ (stop #~(make-kill-destructor)))))))
+
+(define ganeti-noded-service-type
+ (service-type (name 'ganeti-noded)
+ (extensions
+ (list (service-extension shepherd-root-service-type
+ ganeti-noded-service)))
+ (default-value (ganeti-noded-configuration))
+ (description
+ "@command{ganeti-noded} is the daemon which is responsible
+for the node functions in the Ganeti system.")))
+
+(define-record-type* <ganeti-confd-configuration>
+ ganeti-confd-configuration make-ganeti-confd-configuration
+ ganeti-confd-configuration?
+ (ganeti ganeti-confd-configuration-ganeti ;<package>
+ (default ganeti))
+ (port ganeti-confd-configuration-port ;integer
+ (default 1814))
+ (address ganeti-confd-configuration-address ;string
+ (default "0.0.0.0"))
+ (debug? ganeti-confd-configuration-debug? ;Boolean
+ (default #f)))
+
+(define ganeti-confd-service
+ (match-lambda
+ (($ <ganeti-confd-configuration> ganeti port address debug?)
+ (list (shepherd-service
+ (documentation "Run the Ganeti confd daemon.")
+ (provision '(ganeti-confd))
+ (requirement '(user-processes networking))
+ (respawn? #f)
+ (start #~(make-forkexec-constructor
+ (list #$(file-append ganeti "/sbin/ganeti-confd")
+ (string-append "--port=" (number->string #$port))
+ (string-append "--bind=" #$address)
+ #$@(if debug?
+ #~("--debug")
+ #~()))
+ #:environment-variables
+ '#$%default-ganeti-environment-variables
+ #:pid-file "/var/run/ganeti/ganeti-confd.pid"))
+ (stop #~(make-kill-destructor)))))))
+
+(define ganeti-confd-service-type
+ (service-type (name 'ganeti-confd)
+ (extensions
+ (list (service-extension shepherd-root-service-type
+ ganeti-confd-service)))
+ (default-value (ganeti-confd-configuration))
+ (description
+ "@command{ganeti-confd} is a daemon used to answer queries
+related to the configuration of a Ganeti cluster.")))
+
+(define-record-type* <ganeti-wconfd-configuration>
+ ganeti-wconfd-configuration make-ganeti-wconfd-configuration
+ ganeti-wconfd-configuration?
+ (ganeti ganeti-wconfd-configuration-ganeti ;<package>
+ (default ganeti))
+ (no-voting? ganeti-wconfd-configuration-no-voting? ;Boolean
+ (default #f))
+ (debug? ganeti-wconfd-configuration-debug? ;Boolean
+ (default #f)))
+
+(define ganeti-wconfd-service
+ (match-lambda
+ (($ <ganeti-wconfd-configuration> ganeti no-voting? debug?)
+ (list (shepherd-service
+ (documentation "Run the Ganeti wconfd daemon.")
+ (provision '(ganeti-wconfd))
+ (requirement '(user-processes))
+
+ ;; This service will automatically disable itself when not
+ ;; running on the master node. Don't attempt to restart it.
+ (respawn? #f)
+
+ (start #~(make-forkexec-constructor
+ (list #$(file-append ganeti "/sbin/ganeti-wconfd")
+ #$@(if no-voting?
+ #~("--no-voting" "--yes-do-it")
+ #~())
+ #$@(if debug?
+ #~("--debug")
+ #~()))
+ #:environment-variables
+ '#$%default-ganeti-environment-variables
+ #:pid-file "/var/run/ganeti/ganeti-wconfd.pid"))
+ (stop #~(make-kill-destructor)))))))
+
+(define ganeti-wconfd-service-type
+ (service-type (name 'ganeti-wconfd)
+ (extensions
+ (list (service-extension shepherd-root-service-type
+ ganeti-wconfd-service)))
+ (default-value (ganeti-wconfd-configuration))
+ (description
+ "@command{ganeti-wconfd} is the daemon that has authoritative
+knowledge about the configuration and is the only entity that can accept changes
+to it. All jobs that need to modify the configuration will do so by sending
+appropriate requests to this daemon.")))
+
+(define (ganeti-wconfd-forced-service config)
+ (let ((ganeti (ganeti-wconfd-configuration-ganeti config)))
+ (list (shepherd-service
+ (documentation "Forcefully start the Ganeti wconfd daemon (dangerous!).")
+ (provision '(ganeti-wconfd-forced ganeti-wconfd))
+ (requirement '(user-processes))
+ (auto-start? #f)
+ (respawn? #f)
+ (start #~(make-forkexec-constructor
+ (list #$(file-append ganeti "/sbin/ganeti-wconfd")
+ "--force-node" "--no-voting" "--yes-do-it")
+ #:environment-variables
+ '#$%default-ganeti-environment-variables
+ #:pid-file "/var/run/ganeti/ganeti-wconfd.pid"))
+ (stop #~(make-kill-destructor))))))
+
+(define ganeti-wconfd-forced-service-type
+ (service-type (name 'ganeti-force-wconfd)
+ (extensions
+ (list (service-extension shepherd-root-service-type
+ ganeti-wconfd-forced-service)))
+ (default-value (ganeti-wconfd-configuration))
+ (description
+ "This service will forcefully start the wconf daemon even
+on non-master nodes. It is automatically and temporarily started in the event
+of a master-failover. Do not start or enable this service manually unless you
+know exactly what you are doing!")))
+
+(define-record-type* <ganeti-luxid-configuration>
+ ganeti-luxid-configuration make-ganeti-luxid-configuration
+ ganeti-luxid-configuration?
+ (ganeti ganeti-luxid-configuration-ganeti ;<package>
+ (default ganeti))
+ (no-voting? ganeti-luxid-configuration-no-voting? ;Boolean
+ (default #f))
+ (debug? ganeti-luxid-configuration-debug? ;Boolean
+ (default #f)))
+
+(define ganeti-luxid-service
+ (match-lambda
+ (($ <ganeti-luxid-configuration> ganeti no-voting? debug?)
+ (list (shepherd-service
+ (documentation "Run the Ganeti LUXI daemon.")
+ (provision '(ganeti-luxid))
+ (requirement '(user-processes))
+
+ ;; This service will automatically disable itself when not
+ ;; running on the master node. Don't attempt to restart it.
+ (respawn? #f)
+
+ (start #~(make-forkexec-constructor
+ (list #$(file-append ganeti "/sbin/ganeti-luxid")
+ #$@(if no-voting?
+ #~("--no-voting" "--yes-do-it")
+ #~())
+ #$@(if debug?
+ #~("--debug")
+ #~()))
+ #:environment-variables
+ '#$%default-ganeti-environment-variables
+ #:pid-file "/var/run/ganeti/ganeti-luxid.pid"))
+ (stop #~(make-kill-destructor)))))))
+
+(define ganeti-luxid-service-type
+ (service-type (name 'ganeti-luxid)
+ (extensions
+ (list (service-extension shepherd-root-service-type
+ ganeti-luxid-service)))
+ (default-value (ganeti-luxid-configuration))
+ (description
+ "@command{ganeti-luxid} is a daemon used to answer queries
+related to the configuration and the current live state of a Ganeti cluster.
+Additionally, it is the autorative daemon for the Ganeti job queue. Jobs can
+be submitted via this daemon and it schedules and starts them.")))
+
+(define-record-type* <ganeti-rapi-configuration>
+ ganeti-rapi-configuration make-ganeti-rapi-configuration
+ ganeti-rapi-configuration?
+ (ganeti ganeti-rapi-configuration-ganeti ;<package>
+ (default ganeti))
+ (require-authentication?
+ ganeti-rapi-configuration-require-authentication? ;Boolean
+ (default #f))
+ (port ganeti-rapi-configuration-port ;integer
+ (default 5080))
+ (address ganeti-rapi-configuration-address ;string
+ (default "0.0.0.0"))
+ (interface ganeti-rapi-configuration-interface ;string | #f
+ (default #f))
+ (max-clients ganeti-rapi-configuration-max-clients ;integer
+ (default 20))
+ (ssl? ganeti-rapi-configuration-ssl? ;Boolean
+ (default #t))
+ (ssl-key ganeti-rapi-configuration-ssl-key ;string
+ (default "/var/lib/ganeti/server.pem"))
+ (ssl-cert ganeti-rapi-configuration-ssl-cert ;string
+ (default "/var/lib/ganeti/server.pem"))
+ (debug? ganeti-rapi-configuration-debug? ;Boolean
+ (default #f)))
+
+(define ganeti-rapi-service
+ (match-lambda
+ (($ <ganeti-rapi-configuration> ganeti require-authentication? port address
+ interface max-clients ssl? ssl-key ssl-cert
+ debug?)
+ (list (shepherd-service
+ (documentation "Run the Ganeti RAPI daemon.")
+ (provision '(ganeti-rapi))
+ (requirement '(user-processes networking))
+
+ ;; This service will automatically disable itself when not
+ ;; running on the master node. Don't attempt to restart it.
+ (respawn? #f)
+
+ (start #~(make-forkexec-constructor
+ (list #$(file-append ganeti "/sbin/ganeti-rapi")
+ #$@(if require-authentication?
+ #~("--require-authentication")
+ #~())
+ (string-append "--port=" (number->string #$port))
+ (string-append "--bind=" #$address)
+ #$@(if interface
+ #~((string-append "--interface=" #$interface))
+ #~())
+ (string-append "--max-clients="
+ #$(number->string max-clients))
+ #$@(if ssl?
+ #~((string-append "--ssl-key=" #$ssl-key)
+ (string-append "--ssl-cert=" #$ssl-cert))
+ #~("--no-ssl"))
+ #$@(if debug?
+ #~("--debug")
+ #~()))
+ #:environment-variables
+ '#$%default-ganeti-environment-variables
+ #:pid-file "/var/run/ganeti/ganeti-rapi.pid"))
+ (stop #~(make-kill-destructor)))))))
+
+(define ganeti-rapi-service-type
+ (service-type (name 'ganeti-rapi)
+ (extensions
+ (list (service-extension shepherd-root-service-type
+ ganeti-rapi-service)))
+ (default-value (ganeti-rapi-configuration))
+ (description
+ "@command{ganeti-rapi} is the daemon providing a remote API
+for Ganeti clusters.")))
+
+(define-record-type* <ganeti-kvmd-configuration>
+ ganeti-kvmd-configuration make-ganeti-kvmd-configuration
+ ganeti-kvmd-configuration?
+ (ganeti ganeti-kvmd-configuration-ganeti ;<package>
+ (default ganeti))
+ (debug? ganeti-kvmd-configuration-debug? ;Boolean
+ (default #f)))
+
+(define ganeti-kvmd-service
+ (match-lambda
+ (($ <ganeti-kvmd-configuration> ganeti debug?)
+ (list (shepherd-service
+ (documentation "Run the Ganeti KVM daemon.")
+ (provision '(ganeti-kvmd))
+ (requirement '(user-processes))
+
+ ;; This service will automatically disable itself when not
+ ;; needed. Don't attempt to restart it.
+ (respawn? #f)
+
+ (start #~(make-forkexec-constructor
+ (list #$(file-append ganeti "/sbin/ganeti-kvmd")
+ #$@(if debug?
+ #~("--debug")
+ #~()))
+ #:environment-variables
+ '#$%default-ganeti-environment-variables
+ #:pid-file "/var/run/ganeti/ganeti-kvmd.pid"))
+ (stop #~(make-kill-destructor)))))))
+
+(define ganeti-kvmd-service-type
+ (service-type (name 'ganeti-kvmd)
+ (extensions
+ (list (service-extension shepherd-root-service-type
+ ganeti-kvmd-service)))
+ (default-value (ganeti-kvmd-configuration))
+ (description
+ "@command{ganeti-kvmd} is responsible for determining whether
+a given KVM instance was shutdown by an administrator or a user.
+
+The KVM daemon monitors, using @code{inotify}, KVM instances through their QMP
+sockets, which are provided by KVM. Using the QMP sockets, the KVM daemon
+listens for particular shutdown, powerdown, and stop events which will determine
+if a given instance was shutdown by the user or Ganeti, and this result is
+communicated to Ganeti via a special file in the filesystem.")))
+
+(define-record-type* <ganeti-mond-configuration>
+ ganeti-mond-configuration make-ganeti-mond-configuration
+ ganeti-mond-configuration?
+ (ganeti ganeti-mond-configuration-ganeti ;<package>
+ (default ganeti))
+ (port ganeti-mond-configuration-port ;integer | #f
+ (default 80))
+ (address ganeti-mond-configuration-address ;string
+ (default "0.0.0.0"))
+ (debug? ganeti-mond-configuration-debug? ;Boolean
+ (default #f)))
+
+(define ganeti-mond-service
+ (match-lambda
+ (($ <ganeti-mond-configuration> ganeti port address debug?)
+ (list (shepherd-service
+ (documentation "Run the Ganeti monitoring daemon.")
+ (provision '(ganeti-mond))
+ (requirement '(user-processes networking))
+ (respawn? #f)
+ (start #~(make-forkexec-constructor
+ (list #$(file-append ganeti "/sbin/ganeti-mond")
+ (string-append "--port=" (number->string #$port))
+ (string-append "--bind=" #$address)
+ #$@(if debug?
+ #~("--debug")
+ #~()))
+ #:pid-file "/var/run/ganeti/ganeti-mond.pid"))
+ (stop #~(make-kill-destructor)))))))
+
+(define ganeti-mond-service-type
+ (service-type (name 'ganeti-mond)
+ (extensions
+ (list (service-extension shepherd-root-service-type
+ ganeti-mond-service)))
+ (default-value (ganeti-mond-configuration))
+ (description
+ "@command{ganeti-mond} is a daemon providing monitoring
+functionality. It is responsible for running the data collectors and to
+provide the collected information through a HTTP interface.")))
+
+(define-record-type* <ganeti-metad-configuration>
+ ganeti-metad-configuration make-ganeti-metad-configuration
+ ganeti-metad-configuration?
+ (ganeti ganeti-metad-configuration-ganeti ;<package>
+ (default ganeti))
+ (port ganeti-metad-configuration-port ;integer | #f
+ (default 80))
+ (address ganeti-metad-configuration-address ;string | #f
+ (default #f))
+ (debug? ganeti-metad-configuration-debug? ;Boolean
+ (default #f)))
+
+(define ganeti-metad-service
+ (match-lambda
+ (($ <ganeti-metad-configuration> ganeti port address debug?)
+ (list (shepherd-service
+ (documentation "Run the Ganeti metadata daemon.")
+ (provision '(ganeti-metad))
+ (requirement '(user-processes networking))
+ (respawn? #f)
+ (start #~(make-forkexec-constructor
+ (list #$(file-append ganeti "/sbin/ganeti-metad")
+ (string-append "--port=" (number->string #$port))
+ #$@(if address
+ #~((string-append "--bind=" #$address))
+ #~())
+ #$@(if debug?
+ #~("--debug")
+ #~()))
+ #:pid-file "/var/run/ganeti/ganeti-metad.pid"))
+ (stop #~(make-kill-destructor)))))))
+
+(define ganeti-metad-service-type
+ (service-type (name 'ganeti-metad)
+ (extensions
+ (list (service-extension shepherd-root-service-type
+ ganeti-metad-service)))
+ (default-value (ganeti-metad-configuration))
+ (description
+ "@command{ganeti-metad} is an optional daemon that can be
+used to pass information to OS install scripts or instances.")))
+
+(define-record-type* <ganeti-watcher-configuration>
+ ganeti-watcher-configuration make-ganeti-watcher-configuration
+ ganeti-watcher-configuration?
+ (ganeti ganeti-watcher-configuration-ganeti ;<package>
+ (default ganeti))
+ (schedule ganeti-watcher-configuration-schedule ;list | string
+ (default
+ '(next-second-from
+ ;; Run every five minutes.
+ (next-minute (range 0 60 5)))))
+ (rapi-ip ganeti-watcher-configuration-rapi-ip ;#f | string
+ (default #f))
+ (job-age ganeti-watcher-configuration-job-age ;integer
+ (default (* 6 3600)))
+ (verify-disks? ganeti-watcher-configuration-verify-disks? ;Boolean
+ (default #t))
+ (debug? ganeti-watcher-configuration-debug? ;Boolean
+ (default #f)))
+
+(define ganeti-watcher-command
+ (match-lambda
+ (($ <ganeti-watcher-configuration> ganeti _ rapi-ip job-age verify-disks?
+ debug?)
+ #~(lambda ()
+ (system* #$(file-append ganeti "/sbin/ganeti-watcher")
+ #$@(if rapi-ip
+ #~(string-append "--rapi-ip=" #$rapi-ip)
+ #~())
+ (string-append "--job-age=" (number->string #$job-age))
+ #$@(if verify-disks?
+ #~()
+ #~("--no-verify-disks"))
+ #$@(if debug?
+ #~("--debug")
+ #~()))))))
+
+(define (ganeti-watcher-jobs config)
+ (match config
+ (($ <ganeti-watcher-configuration> _ schedule)
+ (list
+ #~(job #$@(match schedule
+ ((? string?)
+ #~(#$schedule))
+ ((? list?)
+ #~('#$schedule)))
+ #$(ganeti-watcher-command config))))))
+
+(define ganeti-watcher-service-type
+ (service-type (name 'ganeti-watcher)
+ (extensions
+ (list (service-extension mcron-service-type
+ ganeti-watcher-jobs)))
+ (default-value (ganeti-watcher-configuration))
+ (description
+ "@command{ganeti-watcher} is a periodically run script that
+performs a number of maintenance actions on the cluster. It will automatically
+restart instances that are marked as ERROR_down, i.e., instances that should be
+running, but are not; and it will also try to repair DRBD links in case a
+secondary node has rebooted. In addition it is responsible for archiving old
+cluster jobs, and it will restart any down Ganeti daemons that are appropriate
+for the current node. If the cluster parameter @code{maintain_node_health} is
+enabled, the watcher will also shutdown instances and DRBD devices if the node
+is declared offline by known master candidates.")))
+
+(define-record-type* <ganeti-cleaner-configuration>
+ ganeti-cleaner-configuration make-ganeti-cleaner-configuration
+ ganeti-cleaner-configuration?
+ (ganeti ganeti-cleaner-configuration-ganeti ;<package>
+ (default ganeti))
+ (master-schedule ganeti-cleaner-configuration-master-schedule ;list | string
+ ;; Run the master cleaner at 01:45 every day.
+ (default "45 1 * * *"))
+ (node-schedule ganeti-cleaner-configuration-node-schedule ;list | string
+ ;; Run the node cleaner at 02:45 every day.
+ (default "45 2 * * *")))
+
+(define ganeti-cleaner-jobs
+ (match-lambda
+ (($ <ganeti-cleaner-configuration> ganeti master-schedule node-schedule)
+ (list
+ #~(job #$@(match master-schedule
+ ((? string?)
+ #~(#$master-schedule))
+ ((? list?)
+ #~('#$master-schedule)))
+ (lambda ()
+ (system* #$(file-append ganeti "/sbin/ganeti-cleaner")
+ "master")))
+ #~(job #$@(match node-schedule
+ ((? string?)
+ #~(#$node-schedule))
+ ((? list?)
+ #~('#$node-schedule)))
+ (lambda ()
+ (system* #$(file-append ganeti "/sbin/ganeti-cleaner")
+ "node")))))))
+
+(define ganeti-cleaner-service-type
+ (service-type (name 'ganeti-cleaner)
+ (extensions
+ (list (service-extension mcron-service-type
+ ganeti-cleaner-jobs)))
+ (default-value (ganeti-cleaner-configuration))
+ (description
+ "@command{ganeti-cleaner} is a script that removes old files
+from the cluster. When called with @code{node} as argument it removes expired
+X509 certificates and keys from @file{/var/run/ganeti/crypto}, as well as
+outdated @command{ganeti-watcher} information.
+
+When called with @code{master} as argument, it instead removes files older
+than 21 days from @file{/var/lib/ganeti/queue/archive}.")))
+
+(define-record-type* <ganeti-configuration>
+ ganeti-configuration make-ganeti-configuration
+ ganeti-configuration?
+ (ganeti ganeti-configuration-ganeti
+ (default ganeti))
+ (noded-configuration ganeti-configuration-noded-configuration
+ (default (ganeti-noded-configuration)))
+ (confd-configuration ganeti-configuration-confd-configuration
+ (default (ganeti-confd-configuration)))
+ (wconfd-configuration ganeti-configuration-wconfd-configuration
+ (default (ganeti-wconfd-configuration)))
+ (luxid-configuration ganeti-configuration-luxid-configuration
+ (default (ganeti-luxid-configuration)))
+ (rapi-configuration ganeti-configuration-rapi-configuration
+ (default (ganeti-rapi-configuration)))
+ (kvmd-configuration ganeti-configuration-kvmd-configuration
+ (default (ganeti-kvmd-configuration)))
+ (mond-configuration ganeti-configuration-mond-configuration
+ (default (ganeti-mond-configuration)))
+ (watcher-configuration ganeti-configuration-watcher-configuration
+ (default (ganeti-watcher-configuration)))
+ (cleaner-configuration ganeti-configuration-cleaner-configuration
+ (default (ganeti-cleaner-configuration)))
+ (file-storage-paths ganeti-configuration-file-storage-paths ;list of strings | gexp
+ (default '()))
+ (os ganeti-configuration-os ;list of <ganeti-os>
+ (default '())))
+
+(define (ganeti-activation config)
+ (with-imported-modules '((guix build utils))
+ #~(begin
+ (use-modules (guix build utils))
+ (for-each mkdir-p
+ '("/var/log/ganeti"
+ "/var/log/ganeti/kvm"
+ "/var/log/ganeti/os"
+ "/var/lib/ganeti/rapi"
+ "/var/lib/ganeti/queue"
+ "/var/lib/ganeti/queue/archive"
+ "/var/run/ganeti/bdev-cache"
+ "/var/run/ganeti/crypto"
+ "/var/run/ganeti/socket"
+ "/var/run/ganeti/instance-disks"
+ "/var/run/ganeti/instance-reason"
+ "/var/run/ganeti/livelocks")))))
+
+(define ganeti-shepherd-services
+ (match-lambda
+ (($ <ganeti-configuration> _ noded confd wconfd luxid rapi kvmd mond)
+ (append (ganeti-noded-service noded)
+ (ganeti-confd-service confd)
+ (ganeti-wconfd-service wconfd)
+ (ganeti-luxid-service luxid)
+ (ganeti-rapi-service rapi)
+ (ganeti-kvmd-service kvmd)
+ (ganeti-mond-service mond)))))
+
+(define ganeti-mcron-jobs
+ (match-lambda
+ (($ <ganeti-configuration> _ _ _ _ _ _ _ _ watcher cleaner)
+ (append (ganeti-watcher-jobs watcher)
+ (ganeti-cleaner-jobs cleaner)))))
+
+(define-record-type* <ganeti-os>
+ ganeti-os make-ganeti-os ganeti-os?
+ (name ganeti-os-name) ;string
+ (variants ganeti-os-variants ;list of <ganeti-os-variant>
+ (default '())))
+
+(define-record-type* <ganeti-os-variant>
+ ganeti-os-variant make-ganeti-os-variant ganeti-os-variant?
+ (name ganeti-os-variant-name) ;string
+ (configuration ganeti-os-variant-configuration)) ;record
+
+(define %default-debootstrap-extra-pkgs
+ ;; Packages suitable for a fully virtualized KVM guest.
+ '("acpi-support-base" "udev" "linux-image-amd64" "openssh-server"
+ "locales-all" "grub-pc"))
+
+(define-record-type* <debootstrap-configuration>
+ debootstrap-configuration make-debootstrap-configuration
+ debootstrap-configuration?
+ ;; This option is treated specially and can be either a gexp or a
+ ;; list of (name . file-like) pairs.
+ (hooks debootstrap-configuration-hooks
+ (default #f))
+ (proxy debootstrap-configuration-proxy (default #f)) ;#f | string
+ (mirror debootstrap-configuration-mirror ;#f | string
+ (default #f))
+ (arch debootstrap-configuration-arch (default #f)) ;#f | string
+ (suite debootstrap-configuration-suite ;#f | string
+ (default "stable"))
+ (extra-pkgs debootstrap-configuration-extra-pkgs ;list of strings
+ (default %default-debootstrap-extra-pkgs))
+ (components debootstrap-configuration-components ;list of strings
+ (default '()))
+ (generate-cache? debootstrap-configuration-generate-cache? ;Boolean
+ (default #t))
+ (clean-cache debootstrap-configuration-clean-cache ;#f | integer
+ (default 14))
+ (partition-style debootstrap-configuration-partition-style ;#f | symbol | string
+ (default 'msdos))
+ (partition-alignment debootstrap-configuration-partition-alignment ;#f | integer
+ (default 2048)))
+
+(define (hooks->directory hooks)
+ (match hooks
+ ((? file-like?)
+ hooks)
+ ((? list?)
+ (let ((names (map car hooks))
+ (files (map cdr hooks)))
+ (with-imported-modules '((guix build utils))
+ (computed-file "hooks-union"
+ #~(begin
+ (use-modules (guix build utils)
+ (ice-9 match))
+ (mkdir-p #$output)
+ (with-directory-excursion #$output
+ (for-each (match-lambda
+ ((name hook)
+ (let ((file-name (string-append
+ #$output "/"
+ (symbol->string name))))
+ ;; Copy to the destination to ensure
+ ;; the file is executable.
+ (copy-file hook file-name)
+ (chmod file-name #o555))))
+ '#$(zip names files))))))))
+ (_ #f)))
+
+(define-gexp-compiler (debootstrap-configuration-compiler
+ (file <debootstrap-configuration>) system target)
+ (match file
+ (($ <debootstrap-configuration> hooks proxy mirror arch suite extra-pkgs
+ components generate-cache? clean-cache
+ partition-style partition-alignment)
+ (let ((customize-dir (hooks->directory hooks)))
+ (gexp->derivation
+ "debootstrap-variant"
+ #~(call-with-output-file (ungexp output "out")
+ (lambda (port)
+ (display
+ (string-append
+ (ungexp-splicing
+ `(,@(if proxy
+ `("PROXY=" ,proxy "\n")
+ '())
+ ,@(if mirror
+ `("MIRROR=" ,mirror "\n")
+ '())
+ ,@(if arch
+ `("ARCH=" ,arch "\n")
+ '())
+ ,@(if suite
+ `("SUITE=" ,suite "\n")
+ '())
+ ,@(if (not (null? extra-pkgs))
+ `("EXTRA_PKGS=" ,(string-join extra-pkgs ",") "\n")
+ '())
+ ,@(if (not (null? components))
+ `("COMPONENTS=" ,(string-join components ",") "\n")
+ '())
+ ,@(if customize-dir
+ `("CUSTOMIZE_DIR=" ,customize-dir "\n")
+ '())
+ ,@(if generate-cache?
+ '("GENERATE_CACHE=yes\n")
+ '("GENERATE_CACHE=no\n"))
+ ,@(if clean-cache
+ `("CLEAN_CACHE=" ,(number->string clean-cache) "\n")
+ '())
+ ,@(if partition-style
+ (if (symbol? partition-style)
+ `("PARTITION_STYLE="
+ ,(symbol->string partition-style) "\n")
+ `("PARTITION_STYLE=" ,partition-style "\n"))
+ '())
+ ,@(if partition-alignment
+ `("PARTITION_ALIGNMENT="
+ ,(number->string partition-alignment) "\n")
+ '()))))
+ port)))
+ #:local-build? #t)))))
+
+(define* (debootstrap-variant name
+ #:optional
+ (configuration (debootstrap-configuration)))
+ (ganeti-os-variant
+ (name name)
+ (configuration configuration)))
+
+(define ganeti-os-variant->configuration
+ (match-lambda
+ (($ <ganeti-os-variant> name configuration)
+ configuration)))
+
+(define (ganeti-os->directory os)
+ "Return the derivation to build the configuration directory to be installed
+in /etc/ganeti/instance-$os for OS."
+ (let* ((name (ganeti-os-name os))
+ (variants (ganeti-os-variants os))
+ (names (map ganeti-os-variant-name variants))
+ (configs (map ganeti-os-variant->configuration variants)))
+ (with-imported-modules '((guix build utils))
+ (define builder
+ #~(begin
+ (use-modules (guix build utils)
+ (ice-9 match)
+ (srfi srfi-1))
+ (mkdir-p #$output)
+ (unless (null? '#$names)
+ (let ((variants-dir (string-append #$output "/variants")))
+ (mkdir-p variants-dir)
+ (call-with-output-file (string-append variants-dir "/variants.list")
+ (lambda (port)
+ (format port (string-join '#$names "\n"))))
+ (for-each (match-lambda
+ ((name file)
+ (symlink file
+ (string-append variants-dir "/" name ".conf"))))
+
+ '#$(zip names configs))))))
+
+ (computed-file (string-append name "-os") builder))))
+
+(define (ganeti-directory file-storage-file os)
+ (let ((dirs (map ganeti-os->directory os))
+ (names (map ganeti-os-name os)))
+ (with-imported-modules '((guix build utils))
+ (define builder
+ #~(begin
+ (use-modules (guix build utils)
+ (ice-9 match))
+ (mkdir-p #$output)
+ (when #$file-storage-file
+ (symlink #$file-storage-file
+ (string-append #$output "/file-storage-paths")))
+ (for-each (match-lambda
+ ((name dest)
+ (symlink dest
+ (string-append #$output "/instance-" name))))
+ '#$(zip names dirs))))
+ (computed-file "etc-ganeti" builder))))
+
+(define (file-storage-file paths)
+ (match paths
+ ((? null?) #f)
+ ((? list?) (plain-file
+ "file-storage-paths"
+ (string-join paths "\n")))
+ (_ paths)))
+
+(define (ganeti-etc-service config)
+ (list `("ganeti" ,(ganeti-directory
+ (file-storage-file
+ (ganeti-configuration-file-storage-paths config))
+ (ganeti-configuration-os config)))))
+
+(define ganeti-service-type
+ (service-type (name 'ganeti)
+ (extensions
+ (list (service-extension activation-service-type
+ ganeti-activation)
+ (service-extension shepherd-root-service-type
+ ganeti-shepherd-services)
+ (service-extension etc-service-type
+ ganeti-etc-service)
+ (service-extension profile-service-type
+ (compose list ganeti-configuration-ganeti))
+ (service-extension mcron-service-type
+ ganeti-mcron-jobs)))
+ (default-value (ganeti-configuration))
+ (description
+ "Ganeti is a family of services that are designed to run
+on a fleet of machines and facilitate deployment and maintenance of virtual
+servers (@dfn{instances}). It can migrate instances between nodes, automatically
+restart failed instances, evacuate nodes, and much more.")))
diff --git a/gnu/tests/virtualization.scm b/gnu/tests/virtualization.scm
index fbdec20805..47415aa701 100644
--- a/gnu/tests/virtualization.scm
+++ b/gnu/tests/virtualization.scm
@@ -1,5 +1,6 @@
;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2017 Christopher Baines <mail <at> cbaines.net>
+;;; Copyright © 2020 Marius Bakke <marius <at> gnu.org>.
;;;
;;; This file is part of GNU Guix.
;;;
@@ -17,6 +18,7 @@
;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
(define-module (gnu tests virtualization)
+ #:use-module (gnu)
#:use-module (gnu tests)
#:use-module (gnu system)
#:use-module (gnu system file-systems)
@@ -24,11 +26,13 @@
#:use-module (gnu services)
#:use-module (gnu services dbus)
#:use-module (gnu services networking)
+ #:use-module (gnu services ssh)
#:use-module (gnu services virtualization)
#:use-module (gnu packages virtualization)
#:use-module (guix gexp)
#:use-module (guix store)
- #:export (%test-libvirt))
+ #:use-module (ice-9 format)
+ #:export (%test-libvirt %test-ganeti-kvm %test-ganeti-lxc))
(define %libvirt-os
(simple-operating-system
@@ -93,3 +97,172 @@
(name "libvirt")
(description "Connect to the running LIBVIRT service.")
(value (run-libvirt-test))))
+
+(define %debootstrap-os-hooks
+ `((test-hook . ,(plain-file "debootstrap-hook"
+ "#!/bin/sh\nexit 0"))))
+(define %ganeti-os
+ (operating-system
+ (host-name "gnt1")
+ (timezone "Europe/Oslo")
+ (locale "en_US.UTF-8")
+
+ (bootloader (bootloader-configuration
+ (bootloader grub-bootloader)
+ (target "/dev/vda")))
+ (file-systems (cons (file-system
+ (device (file-system-label "my-root"))
+ (mount-point "/")
+ (type "ext4"))
+ %base-file-systems))
+ (firmware '())
+
+ ;; The hosts file must contain a nonlocal IP for host-name.
+ ;; In addition, the cluster name must resolve to an IP address that
+ ;; is not currently provisioned.
+ (hosts-file (plain-file "hosts" (format #f "
+127.0.0.1 localhost
+10.0.2.2 gnt1
+192.168.254.254 ganeti.local
+")))
+
+ (packages (append (list ganeti-instance-debootstrap)
+ %base-packages))
+ (services
+ (append (list (static-networking-service "eth0" "10.0.2.2"
+ #:netmask "255.255.255.0"
+ #:gateway "10.0.2.1"
+ #:name-servers '("10.0.2.1"))
+
+ (service openssh-service-type
+ (openssh-configuration
+ (permit-root-login 'without-password)))
+
+ (service ganeti-service-type
+ (ganeti-configuration
+ (file-storage-paths
+ '("/srv/ganeti/file-storage"))
+ (os
+ (list (ganeti-os
+ (name "debootstrap")
+ (variants
+ (list (debootstrap-variant
+ "buster"
+ (debootstrap-configuration
+ (hooks %debootstrap-os-hooks)))))))))))
+ %base-services))))
+
+(define* (run-ganeti-test hypervisor #:key
+ (master-netdev "eth0")
+ (hvparams '())
+ (extra-packages '()))
+ "Run tests in %GANETI-OS."
+ (define os
+ (marionette-operating-system
+ (operating-system
+ (inherit %ganeti-os)
+ (packages (append extra-packages
+ (operating-system-packages %ganeti-os))))
+ #:imported-modules '((gnu services herd)
+ (guix combinators))))
+
+ (define vm
+ (virtual-machine
+ (operating-system os)
+ ;; Some of the daemons are fairly memory-hungry.
+ (memory-size 512)
+ (port-forwardings '((5080 . 5080)))))
+
+ (define test
+ (with-imported-modules '((gnu build marionette))
+ #~(begin
+ (use-modules (srfi srfi-11) (srfi srfi-64)
+ (gnu build marionette))
+
+ (define marionette
+ (make-marionette (list #$vm)))
+
+ (mkdir #$output)
+ (chdir #$output)
+
+ (test-begin "ganeti")
+
+ ;; Ganeti uses the Shepherd to start/stop daemons, so make sure
+ ;; it is ready before we begin. It takes a while because all
+ ;; Ganeti daemons fail to start initially.
+ (test-assert "shepherd is ready"
+ (wait-for-unix-socket "/var/run/shepherd/socket" marionette))
+
+ (test-eq "gnt-cluster init"
+ 0
+ (marionette-eval
+ '(begin
+ (setenv
+ "PATH"
+ ;; Init needs to run 'ssh-keygen', 'ip', etc.
+ "/run/current-system/profile/sbin:/run/current-system/profile/bin")
+ (system* #$(file-append ganeti "/sbin/gnt-cluster") "init"
+ (string-append "--master-netdev=" #$master-netdev)
+ ;; TODO: Enable more disk backends.
+ "--enabled-disk-templates=file"
+ (string-append "--enabled-hypervisors="
+ #$hypervisor)
+ ;; Set kernel_path to an empty string to prevent
+ ;; 'gnt-cluster verify' from testing for its presence.
+ (string-append "--hypervisor-parameters="
+ #$hypervisor ":"
+ (string-join '#$hvparams "\n"))
+ ;; Set the default NIC mode to 'routed' to avoid having to
+ ;; configure a full bridge to placate 'gnt-cluster verify'.
+ "--nic-parameters=mode=routed,link=eth0"
+ "ganeti.local"))
+ marionette))
+
+ (test-eq "gnt-cluster verify"
+ 0
+ (marionette-eval
+ '(begin
+ (system* #$(file-append ganeti "/sbin/gnt-cluster") "verify"))
+ marionette))
+
+ (test-equal "gnt-os list"
+ "debootstrap+buster"
+ (marionette-eval
+ '(begin
+ (use-modules (ice-9 popen) (ice-9 rdelim))
+ (let* ((port (open-pipe*
+ OPEN_READ
+ (string-append #$ganeti "/sbin/gnt-os")
+ "list" "--no-headers"))
+ (output (read-line port)))
+ (close-pipe port)
+ output))
+ marionette))
+
+ (test-eq "gnt-cluster destroy"
+ 0
+ (marionette-eval
+ '(begin
+ (system* #$(file-append ganeti "/sbin/gnt-cluster")
+ "destroy" "--yes-do-it"))
+ marionette))
+
+ (test-end)
+ (exit (= (test-runner-fail-count (test-runner-current)) 0)))))
+
+ (gexp->derivation (string-append "ganeti-" hypervisor "-test") test))
+
+(define %test-ganeti-kvm
+ (system-test
+ (name "ganeti-kvm")
+ (description "Provision a Ganeti cluster using the KVM hypervisor.")
+ (value (run-ganeti-test "kvm"
+ #:hvparams '("kernel_path=")
+ #:extra-packages (list qemu)))))
+
+(define %test-ganeti-lxc
+ (system-test
+ (name "ganeti-lxc")
+ (description "Provision a Ganeti cluster using LXC as the hypervisor.")
+ (value (run-ganeti-test "lxc"
+ #:extra-packages (list lxc)))))
--
2.27.0
guix-patches <at> gnu.org:bug#42261; Package guix-patches.
(Wed, 08 Jul 2020 10:13:02 GMT) Full text and rfc822 format available.Message #20 received at 42261 <at> debbugs.gnu.org (full text, mbox):
From: Marius Bakke <marius <at> gnu.org> To: 42261 <at> debbugs.gnu.org Cc: Marius Bakke <mbakke <at> fastmail.com> Subject: [PATCH] website: Add draft of a Ganeti cluster post. Date: Wed, 8 Jul 2020 12:11:18 +0200
From: Marius Bakke <mbakke <at> fastmail.com>
* website/drafts/ganeti-cluster-on-guix.md: New file.
---
website/drafts/ganeti-cluster-on-guix.md | 414 +++++++++++++++++++++++
1 file changed, 414 insertions(+)
create mode 100644 website/drafts/ganeti-cluster-on-guix.md
diff --git a/website/drafts/ganeti-cluster-on-guix.md b/website/drafts/ganeti-cluster-on-guix.md
new file mode 100644
index 0000000..253681b
--- /dev/null
+++ b/website/drafts/ganeti-cluster-on-guix.md
@@ -0,0 +1,414 @@
+title: Running a Ganeti cluster on Guix
+date: 2020-07-10 12:00
+author: Marius Bakke
+tags: Virtualization, Ganeti
+---
+The latest addition to Guix's ever-growing list of services is a little-known
+virtualization toolkit called [Ganeti](http://www.ganeti.org/). Ganeti is
+designed to keep virtual machines running on a cluster of servers even in the
+event of hardware failures, and to make maintenance and recovery tasks easy.
+
+It is comparable to tools such as
+[Proxmox](https://www.proxmox.com/en/proxmox-ve) or
+[oVirt](https://www.ovirt.org/), but has some distinctive features. One is
+that there is no GUI: [third](https://github.com/osuosl/ganeti_webmgr)
+[party](https://github.com/grnet/ganetimgr)
+[ones](https://github.com/sipgate/ganeti-control-center) exist, but are not
+currently packaged in Guix, so you are left with a rich command-line client
+and a fully featured
+[remote API](http://docs.ganeti.org/ganeti/master/html/rapi.html).
+
+Another interesting feature is that installing Ganeti on its own leaves you
+no way to actually deploy any virtual machines. That probably sounds crazy,
+but stems from the fact that Ganeti is designed to be API-driven and automated,
+thus it comes with a
+[OS API](http://docs.ganeti.org/ganeti/master/html/man-ganeti-os-interface.html)
+and users need to install one or more *OS providers* in addition to Ganeti.
+OS providers offer a declarative way to deploy virtual machine variants and
+should feel natural to Guix users. At the time of writing, the providers
+available in Guix are [debootstrap](https://github.com/ganeti/instance-debootstrap)
+for provisioning Debian- and Ubuntu-based VMs, and of course a
+[Guix](https://github.com/mbakke/ganeti-instance-guix) provider.
+
+Finally Ganeti comes with a sophisticated scheduler that efficiently packs
+virtual machines across a cluster while maintaining N+1 redundancy in case
+of a failover scenario. It can also make informed scheduling decisions
+based on various cluster tags, such as ensuring primary and secondary nodes
+are on different power distribution lines.
+
+(Note: if you are looking for a way to run just a few virtual machines on
+your local computer, you are probably better off using
+[libvirt](https://guix.gnu.org/manual/en/guix.html#index-libvirt) or even
+a [Childhurd](https://guix.gnu.org/manual/devel/en/guix.html#index-hurd_002dvm_002dservice_002dtype), as Ganeti is fairly heavyweight and requires a complicated networking
+setup.)
+
+
+# Preparing the configuration
+
+With introductions out of the way, let's see how we can deploy a Ganeti
+cluster using Guix. For this tutorial we will create a two-node cluster
+and connect instances to the local network using an
+[Open vSwitch](https://www.openvswitch.org/) bridge with no VLANs. We assume
+that each node has a single network interface named `eth0` connected to the
+same network, and that a dedicated partition `/dev/sdz3` is available for
+virtual machine storage. It is possible to store VMs on a number of other
+storage backends, but a dedicated drive (or rather LVM volume group) is
+necessary to use the [DRBD](https://www.linbit.com/drbd/) integration to
+replicate VM disks.
+
+We'll start off by defining a few helper services to create the Open vSwitch
+bridge and ensure the physical network interface is in the "up" state. Since
+Open vSwich stores the configuration in a database, you might as well run the
+equivalent `ovs-vsctl` commands on the host once and be done with it, but we
+do it through the configuration system to ensure we don't forget it in the
+future when adding or reinstalling nodes.
+
+```
+(define (start-interface if)
+ #~(let ((ip (string-append #$iproute "/sbin/ip")))
+ (invoke/quiet ip "link" "set" #$if "up")))
+
+(define (stop-interface if)
+ #~(let ((ip (string-append #$iproute "/sbin/ip")))
+ (invoke/quiet ip "link" "set" #$if "down")))
+
+;; This service is necessary to ensure eth0 is in the "up" state on boot
+;; since it is otherwise unmanaged from Guix PoV.
+(define (ifup-service if)
+ (let ((name (string-append "ifup-" if)))
+ (simple-service name shepherd-root-service-type
+ (list (shepherd-service
+ (provision (list (string->symbol name)))
+ (start #~(lambda ()
+ #$(start-interface if)))
+ (stop #~(lambda ()
+ #$(stop-interface if)))
+ (respawn? #f))))))
+
+(define* (create-openvswitch-bridge bridge uplink
+ #:key (vlan-mode #f))
+ #~(let ((ovs-vsctl (lambda (cmd)
+ (apply invoke/quiet
+ #$(file-append openvswitch "/bin/ovs-vsctl")
+ (string-tokenize cmd)))))
+ (and (ovs-vsctl (string-append "--may-exist add-br " #$bridge))
+ (ovs-vsctl (string-append "--may-exist add-port " #$bridge " "
+ #$uplink
+ (if #$vlan_mode
+ (format #f " vlan_mode=~a " #$vlan-mode)
+ ""))))))
+
+(define* (create-openvswitch-internal-port bridge port
+ #:key (vlan-mode #f))
+ #~(invoke/quiet #$(file-append openvswitch "/bin/ovs-vsctl")
+ "--may-exist" "add-port" #$bridge #$port
+ (if #$vlan_mode
+ (string-append "vlan_mode=" #$vlan-mode)
+ "")
+ "--" "set" "Interface" #$port "type=internal"))
+
+(define %openvswitch-configuration-service
+ (simple-service 'openvswitch-configuration shepherd-root-service-type
+ (list (shepherd-service
+ (provision '(openvswitch-configuration))
+ (requirement '(vswitchd))
+ (start #~(lambda ()
+ #$(create-openvswitch-bridge
+ "br0" "eth0"
+ #:vlan_mode "native-untagged")
+ #$(create-openvswitch-internal-port
+ "br0" "gnt0"
+ #:vlan_mode "native-untagged")))
+ (respawn? #f)))))
+```
+
+This defines a `openvswitch-configuration` service object that creates a
+logical switch `br0`, connects `eth0` as the "uplink", and creates a logical
+port `gnt0` that we will use later as the main network interface for this
+system. We also create an `ifup` service that can bring network interfaces
+up and down. By themselves these variables do nothing, we also have to add
+them to our `operating-system` configuration below.
+
+A configuration like this might be suitable for a small home network. In most
+"real world" deployments you would use tagged VLANs, and maybe a traditional
+Linux bridge instead of Open vSwitch. You can also forego bridging altogether
+with a `routed` networking setup, or do any combination of the three.
+
+With this in place, we can start creating the `operating-system` configuration
+that we will use for the Ganeti servers:
+
+```
+(operating-system
+ (host-name "node1")
+ [...]
+ ;; Ganeti requires that each node and the cluster address resolves to an
+ ;; IP address. The easiest way to achieve this is by adding everything
+ ;; to the hosts file.
+ (hosts-file (plain-file "hosts" (format #f "\
+127.0.0.1 localhost
+::1 localhost
+
+192.168.1.101 node1
+192.168.1.102 node2
+192.168.1.254 ganeti.lan
+")))
+ (kernel-arguments
+ (append %default-kernel-arguments
+ '(;; Disable DRBDs usermode helper, as Ganeti
+ ;; is the only thing that should manage DRBD.
+ "drbd.usermode_helper=/run/current-system/profile/bin/true")))
+
+ (packages (append (map specification->package
+ '("qemu" "drbd-utils" "lvm2"
+ "ganeti-instance-guix"
+ "ganeti-instance-debootstrap"))
+ %base-packages))
+
+ (services (cons* (service ganeti-service-type
+ (ganeti-configuration
+ (file-storage-paths '("/srv/ganeti/file-storage"))
+ (os
+ (list (ganeti-os
+ (name "debootstrap")
+ (variants
+ (list (debootstrap-variant
+ "buster"
+ (debootstrap-configuration
+ (hooks
+ (local-file
+ "debootstrap-hooks"
+ #:recursive? #t))))
+ (debootstrap-variant
+ "testing+contrib"
+ (debootstrap-configuration
+ (suite "testing")
+ (components '("main" "contrib")))))))))))
+
+ ;; Create a static IP on the "gnt0" Open vSwitch interface.
+ (service openvswitch-service-type)
+ %openvswitch-configuration-service
+ (ifup-service "eth0")
+ (static-networking-service "gnt0" "192.168.1.101"
+ #:netmask "255.255.255.0"
+ #:gateway "192.168.1.1"
+ #:requirement '(openvswitch-configuration)
+ #:name-servers '("192.168.1.1"))
+
+ ;; Ganeti needs SSH to communicate between nodes.
+ (service openssh-service-type
+ (openssh-configuration
+ (permit-root-login 'without-password)))
+ %base-services)))
+```
+
+Debootstrap variants rely on a set of scripts (known as "hooks") in the
+installation process to do things like configure networking, install bootloader,
+create users, etc. In the example above, the "buster" variant will use a local
+directory next to the configuration file named "debootstrap-hooks" (it is copied
+into the final system closure), whereas the "testing+contrib" variant has no hooks
+defined and will use `/etc/ganeti/instance-debootstrap/hooks` if it exists.
+
+Ganeti veterans may be surprised that each OS variant has its own hooks. All
+Ganeti clusters I know of use a single set of hooks for all variants, sometimes
+with additional logic inside the script based on the variant. Guix offers a
+powerful abstraction that makes it trivial to create per-variant hooks, obsoleting
+the need for a big `/etc/ganeti/instance-debootstrap/hooks` directory. Of course
+you can still create it using `extra-special-file` and leave the `hooks` property
+of the variants as `#f`.
+
+Not all Ganeti options are exposed in the configuration system yet. If you
+find it limiting, you can add custom files using `extra-special-file`, or
+ideally extend the `<ganeti-configuration>` data type to suite your needs.
+Of course you can use `gnt-cluster copyfile` and `gnt-cluster command`
+to distribute files or run executables, but beware that undeclared changes
+in `/etc` may be lost on the next reboot or reconfigure.
+
+
+# Initializing a cluster
+
+At this stage, you should run `guix system reconfigure` with the new
+configuration on all nodes that will participate in the cluster. If you
+do this over SSH or with
+[guix deploy](https://guix.gnu.org/blog/2019/managing-servers-with-gnu-guix-a-tutorial/),
+beware that `eth0` will lose network connectivity once it is "plugged in to"
+the virtual switch, and you need to add any IP configuration to `gnt0`.
+
+The Guix configuration system does not currently support declaring LVM
+volume groups, so we will create these manually on each node. We could
+write our own declarative configuration like the `ifup-service`, but for
+brevity and safety reasons we'll do it "by hand":
+
+```
+pvcreate /dev/sdz3
+vgcreate ganetivg /dev/sdz3
+```
+
+On the node that will act as the "master node", run the init command:
+
+```
+gnt-cluster init \
+ --master-netdev=gnt0 \
+ --vg-name=ganetivg \
+ --enabled-disk-templates=file,plain,drbd \
+ --drbd-usermode-helper=/run/current-system/profile/bin/true \
+ --enabled-hypervisors=kvm \
+ --no-etc-hosts \
+ --no-ssh-init \
+ ganeti.lan
+```
+
+If you are okay with Ganeti taking control over SSH `authorized_keys` and
+`known_hosts`, remove the `--no-ssh-init` option. Guix users might prefer
+to manage the relevant files using `openssh-configuration`. All nodes in
+the cluster must be able to reach each other over SSH as the root user.
+
+Similarly, Ganeti can update the `/etc/hosts` file when nodes are added or
+removed, but it makes little sense on Guix as it is recreated every reboot.
+
+If all goes well, the command returns no output and you should have the
+`ganeti.lan` IP address visible on `gnt0`. You can run `gnt-cluster verify`
+to check that the cluster is in good shape. Most likely it complains about
+something:
+
+```
+# TODO
+```
+
+Use `gnt-cluster modify` to change the running state of the cluster:
+
+```
+gnt-cluster modify -H kvm:kernel_path=
+```
+
+The command above removes the warning about the default KVM kernel being
+missing, making `gnt-cluster verify` happy. For this tutorial we only use
+fully virtualized instances, but users might want to set `kernel_path` to a
+suitable VM kernel.
+
+Now let's add our other machine to the cluster:
+
+```
+gnt-node add node2
+```
+
+Ganeti will log into the node, copy the cluster configuration and start the
+relevant Shepherd services. No output means the command succeeded. Run
+`gnt-cluster verify` again to check that everything is in order:
+
+```
+gnt-cluster verify
+```
+
+If you get warnings about SSH authorizations here, you should fix those
+before proceeding. If you used `--no-ssh-init` earlier you may need to
+update `/var/lib/ganeti/known_hosts` with the new node information, either
+with `gnt-cluster copyfile` or by adding it to the OS configuration.
+
+The above configuration will make three operating systems available:
+
+```
+# gnt-os list
+Name
+guix
+debootstrap+buster
+debootstrap+testing+contrib
+```
+
+Let's try them out. But first we'll make Ganeti aware of our network
+so it can choose a static IP for the virtual machines.
+
+```
+# gnt-network add --network=192.168.1.0/24 --gateway=192.168.1.1 lan
+# gnt-network connect -N mode=openvswitch,link=br0 lan
+```
+
+Now we can add an instance:
+
+```
+gnt-instance add --no-name-check --no-ip-check -o debootstrap+buster \
+ -t drbd --disk 0:size=5G -B memory=256m,vcpus=2 \
+ --net 0:network=lan,ip=pool bustervm1
+```
+
+Ganeti will automatically select the optimal primary and secondary node
+for this VM based on available cluster resources. You can manually
+specify primary and secondary nodes with the `-n` and `-s` options.
+
+By default Ganeti assumes that the new instance is already configured in DNS,
+so we need `--no-name-check` and `--no-ip-check` to bypass some sanity tests.
+
+Try adding another instance, now using the Guix OS provider:
+
+```
+gnt-instance add --no-name-check --no-ip-check -o guix \
+ -t plain --disk 0:size=5G -B memory=1G,vcpus=4 \
+ --net 0:network=lan,ip=pool guix1
+```
+
+The Guix OS has a built-in configuration that starts an SSH server and authorizes
+the hosts SSH key, and configures static networking based on information from
+Ganeti. It is possible to specify a custom configuration file, and even a
+specific Guix commit:
+
+```
+gnt-instance add --no-name-check --no-ip-check -o guix \
+ -t file --file-storage-dir=/srv/ganeti/file-storage \
+ --disk 0:size=20G -B memory=4G,vpus=3 \
+ --net 0:network=lan,ip=pool \
+ -O "config=$(base64 /the/config/file.scm),commit=<commit>" \
+ custom-guix
+```
+
+That's it for this tutorial! If you are new to Ganeti, you should
+familiarize yourself with the `gnt-` family commands. Fun stuff to
+do include `gnt-instance migrate` to move VMs between hosts,
+`gnt-node evacuate` to migrate _all_ VMs off a node, and
+`gnt-cluster master-failover` to move the master role to a different node.
+
+
+# Final remarks
+
+Like most services in Guix, Ganeti comes with a
+[system test](https://guix.gnu.org/blog/2016/guixsd-system-tests/)
+that [runs in a VM](FIXME) and ensures that things like initializing a cluster
+work. The continuous integration system
+[runs this automatically](https://ci.guix.gnu.org/search?query=ganeti), and
+users can run it locally with `make check-system TESTS=ganeti`. Such
+tests give us confidence that both the package and configuration system work,
+and allows rapid testing of the configuration API. Currently it does little
+more than `gnt-cluster verify`, but it can be extended to provision a real
+cluster inside Ganeti and try things like live migration.
+
+The author had a lot of fun creating
+[native data types](FIXME manual link)
+in the Guix configuration system for the Ganeti OS specification. The API
+went through at least three major revisions during the writing of this blog
+post. There is still room for improvement, but I decided I had to stop
+tweaking it and instead focus on shipping the thing. Feedback welcome!
+
+Having OS support in the configuration system lets us benefit from Guix's
+provenance tracking and we can easily `guix system roll-back` any breaking
+changes. Ganeti is usually coupled with tools such as Puppet or SaltStack to
+keep things in sync between nodes, but that should not be necessary here.
+
+So far only the `KVM` hypervisor has been tested. If you use LXC or Xen with
+Ganeti, please reach out to `guix-devel <at> gnu.org` and share your experience.
+
+#### About GNU Guix
+
+[GNU Guix](https://guix.gnu.org) is a transactional package
+manager and an advanced distribution of the GNU system that [respects
+user
+freedom](https://www.gnu.org/distros/free-system-distribution-guidelines.html).
+Guix can be used on top of any system running the kernel Linux, or it
+can be used as a standalone operating system distribution for i686,
+x86_64, ARMv7, and AArch64 machines.
+
+In addition to standard package management features, Guix supports
+transactional upgrades and roll-backs, unprivileged package management,
+per-user profiles, and garbage collection. When used as a standalone
+GNU/Linux distribution, Guix offers a declarative, stateless approach to
+operating system configuration management. Guix is highly customizable
+and hackable through [Guile](https://www.gnu.org/software/guile)
+programming interfaces and extensions to the
+[Scheme](http://schemers.org) language.
--
2.27.0
guix-patches <at> gnu.org:bug#42261; Package guix-patches.
(Fri, 10 Jul 2020 20:59:01 GMT) Full text and rfc822 format available.Message #23 received at 42261 <at> debbugs.gnu.org (full text, mbox):
From: Ludovic Courtès <ludo <at> gnu.org> To: Marius Bakke <marius <at> gnu.org> Cc: 42261 <at> debbugs.gnu.org Subject: Re: [bug#42261] [PATCH 4/4] services: Add ganeti. Date: Fri, 10 Jul 2020 22:58:29 +0200
Hello!
Marius Bakke <marius <at> gnu.org> skribis:
> * gnu/services/virtualization.scm (<ganeti-noded-configuration>,
> <ganeti-confd-configuration>, <ganeti-wconfd-configuration>,
> <ganeti-luxid-configuration>, <ganeti-rapi-configuration>,
> <ganeti-kvmd-configuration>, <ganeti-mond-configuration>),
> <ganeti-metad-configuration>, <ganeti-watcher-configuration>,
> <ganeti-cleaner-configuration>, <ganeti-configuration>, <ganeti-os>,
> <ganeti-os-variant>, <debootstrap-configuration>): New record types.
> (%default-ganeti-environment-variables, ganeti-noded-service,
> ganeti-noded-service-type, ganeti-confd-service, ganeti-confd-service-type,
> ganeti-wconfd-service, ganeti-wconfd-service-type, ganeti-luxid-service,
> ganeti-luxid-service-type, ganeti-rapi-service, ganeti-rapi-service-type,
> ganeti-kvmd-service, ganeti-kvmd-service-type, ganeti-mond-service,
> ganeti-mond-service-type, ganeti-metad-service, ganeti-metad-service-type,
> ganeti-watcher-command, ganeti-watcher-jobs, ganeti-watcher-service-type,
> ganeti-cleaner-jobs, ganeti-cleaner-service-type, ganeti-activation,
> ganeti-shepherd-services, ganeti-mcron-jobs, ganeti-service-type,
> hooks->directory, debootstrap-configuration-compiler, debootstrap-variant,
> ganeti-os-variant->configuration, ganeti-os->directory, ganeti-directory,
> file-storage-file, ganeti-etc-service, ganeti-service-type): New variables.
> * gnu/tests/virtualization.scm (%debootstrap-hooks, %ganeti-os,
> run-ganeti-test, %test-ganeti-kvm, %test-ganeti-lxc): New variables.
> * doc/guix.texi (Virtualization Services): Document accordingly.
Since it’s a big chunk, perhaps it could live in (gnu services ganeti)?
> +Ganeti is a cluster-based virtual machine management system. It consists
^
Maybe add one more sentence to give an idea of what it does or what
features it provides.
> +of multiple services which are described later in this section. In addition
> +to the Ganeti service, you will need the OpenSSH service
> +(@pxref{Networking Services, @code{openssh-service-type}}), and update the
> +@file{/etc/hosts} file (@pxref{operating-system Reference, @code{hosts-file}})
> +with the cluster name and address (or use a DNS server). Here is an example
> +configuration for a Ganeti cluster node:
^
Add “that does X and Y”, or “with X nodes running Y”, something like
that. :-)
> +There is also a
> +@url{https://guix.gnu.org/blog/2020/ganeti-cluster-on-guix/,blog post}
> +describing how to configure a small cluster.
It’d be great to see if part of the examples in the post (which I
haven’t read yet) can be folded in the manual.
> +@table @asis
> +@item @code{ganeti} (default: @code{ganeti})
> +The @code{ganeti} package to use. It will be installed to the system profile
> +and make @command{gnt-cluster}, @command{gnt-instance}, etc available. Note
> +that the value specified here does not affect the other services as each refer
> +to a specific @code{ganeti} package (see below).
> +
> +@item @code{noded-configuration} (default: @code{(ganeti-noded-configuration)})
> +@item @code{confd-configuration} (default: @code{(ganeti-confd-configuration)})
> +@item @code{wconfd-configuration} (default: @code{(ganeti-wconfd-configuration)})
> +@item @code{luxid-configuration} (default: @code{(ganeti-luxid-configuration)})
> +@item @code{rapi-configuration} (default: @code{(ganeti-rapi-configuration)})
> +@item @code{kvmd-configuration} (default: @code{(ganeti-kvmd-configuration)})
> +@item @code{mond-configuration} (default: @code{(ganeti-mond-configuration)})
> +@item @code{watcher-configuration} (default: @code{(ganeti-watcher-configuration)})
> +@item @code{cleaner-configuration} (default: @code{(ganeti-cleaner-configuration)})
You need @itemx for all but the first one.
Anyway, that looks very nice!
Ludo’.
guix-patches <at> gnu.org:bug#42261; Package guix-patches.
(Fri, 10 Jul 2020 21:02:02 GMT) Full text and rfc822 format available.Message #26 received at 42261 <at> debbugs.gnu.org (full text, mbox):
From: Ludovic Courtès <ludo <at> gnu.org> To: Marius Bakke <marius <at> gnu.org> Cc: 42261 <at> debbugs.gnu.org Subject: Re: [bug#42261] [PATCH 1/4] gnu: Add ganeti. Date: Fri, 10 Jul 2020 23:00:40 +0200
Marius Bakke <marius <at> gnu.org> skribis: > * gnu/packages/virtualization.scm (system->qemu-target, ganeti): New variables. > * gnu/packages/patches/ganeti-copy-hmac.patch, > gnu/packages/patches/ganeti-disable-version-symlinks.patch, > gnu/packages/patches/ganeti-drbd-compat.patch, > gnu/packages/patches/ganeti-haskell-pythondir.patch, > gnu/packages/patches/ganeti-openvswitch-may-exist.patch, > gnu/packages/patches/ganeti-preserve-PYTHONPATH.patch, > gnu/packages/patches/ganeti-shepherd-master-failover.patch, > gnu/packages/patches/ganeti-shepherd-support.patch: New files. > * gnu/local.mk (dist_patch_DATA): Adjust accordingly. > --- > gnu/local.mk | 8 + > gnu/packages/patches/ganeti-copy-hmac.patch | 83 ++++ > .../ganeti-disable-version-symlinks.patch | 136 +++++++ > gnu/packages/patches/ganeti-drbd-compat.patch | 168 ++++++++ > .../patches/ganeti-haskell-pythondir.patch | 66 +++ > .../ganeti-openvswitch-may-exist.patch | 25 ++ > .../patches/ganeti-preserve-PYTHONPATH.patch | 21 + > .../ganeti-shepherd-master-failover.patch | 26 ++ > .../patches/ganeti-shepherd-support.patch | 87 ++++ > gnu/packages/virtualization.scm | 378 ++++++++++++++++++ > 10 files changed, 998 insertions(+) The patches are quite big, but maybe that’s unavoidable. Apart from that, on a cursory look it LGTM! Ludo’.
guix-patches <at> gnu.org:bug#42261; Package guix-patches.
(Fri, 10 Jul 2020 21:07:02 GMT) Full text and rfc822 format available.Message #29 received at 42261 <at> debbugs.gnu.org (full text, mbox):
From: Ludovic Courtès <ludo <at> gnu.org> To: Marius Bakke <marius <at> gnu.org> Cc: Marius Bakke <mbakke <at> fastmail.com>, 42261 <at> debbugs.gnu.org Subject: Re: [bug#42261] [PATCH] website: Add draft of a Ganeti cluster post. Date: Fri, 10 Jul 2020 23:05:43 +0200
Hey! Marius Bakke <marius <at> gnu.org> skribis: > From: Marius Bakke <mbakke <at> fastmail.com> > > * website/drafts/ganeti-cluster-on-guix.md: New file. Very nice! There’s a couple of FIXME/TODO links that you’ll have to address, but other than that I found it interesting and pleasant to read. > +(Note: if you are looking for a way to run just a few virtual machines on > +your local computer, you are probably better off using > +[libvirt](https://guix.gnu.org/manual/en/guix.html#index-libvirt) or even > +a [Childhurd](https://guix.gnu.org/manual/devel/en/guix.html#index-hurd_002dvm_002dservice_002dtype), as Ganeti is fairly heavyweight and requires a complicated networking > +setup.) Thumbs up for the Hurd plug. :-) Ludo’.
guix-patches <at> gnu.org:bug#42261; Package guix-patches.
(Sat, 11 Jul 2020 21:55:01 GMT) Full text and rfc822 format available.Message #32 received at 42261 <at> debbugs.gnu.org (full text, mbox):
From: Marius Bakke <marius <at> gnu.org> To: Ludovic Courtès <ludo <at> gnu.org> Cc: 42261 <at> debbugs.gnu.org Subject: Re: [bug#42261] [PATCH 4/4] services: Add ganeti. Date: Sat, 11 Jul 2020 23:54:29 +0200
[Message part 1 (text/plain, inline)]
Hi!
Thanks a bunch for reading through this. :-)
Ludovic Courtès <ludo <at> gnu.org> writes:
> Hello!
>
> Marius Bakke <marius <at> gnu.org> skribis:
>
>> * gnu/services/virtualization.scm (<ganeti-noded-configuration>,
>> <ganeti-confd-configuration>, <ganeti-wconfd-configuration>,
>> <ganeti-luxid-configuration>, <ganeti-rapi-configuration>,
>> <ganeti-kvmd-configuration>, <ganeti-mond-configuration>),
>> <ganeti-metad-configuration>, <ganeti-watcher-configuration>,
>> <ganeti-cleaner-configuration>, <ganeti-configuration>, <ganeti-os>,
>> <ganeti-os-variant>, <debootstrap-configuration>): New record types.
>> (%default-ganeti-environment-variables, ganeti-noded-service,
>> ganeti-noded-service-type, ganeti-confd-service, ganeti-confd-service-type,
>> ganeti-wconfd-service, ganeti-wconfd-service-type, ganeti-luxid-service,
>> ganeti-luxid-service-type, ganeti-rapi-service, ganeti-rapi-service-type,
>> ganeti-kvmd-service, ganeti-kvmd-service-type, ganeti-mond-service,
>> ganeti-mond-service-type, ganeti-metad-service, ganeti-metad-service-type,
>> ganeti-watcher-command, ganeti-watcher-jobs, ganeti-watcher-service-type,
>> ganeti-cleaner-jobs, ganeti-cleaner-service-type, ganeti-activation,
>> ganeti-shepherd-services, ganeti-mcron-jobs, ganeti-service-type,
>> hooks->directory, debootstrap-configuration-compiler, debootstrap-variant,
>> ganeti-os-variant->configuration, ganeti-os->directory, ganeti-directory,
>> file-storage-file, ganeti-etc-service, ganeti-service-type): New variables.
>> * gnu/tests/virtualization.scm (%debootstrap-hooks, %ganeti-os,
>> run-ganeti-test, %test-ganeti-kvm, %test-ganeti-lxc): New variables.
>> * doc/guix.texi (Virtualization Services): Document accordingly.
>
> Since it’s a big chunk, perhaps it could live in (gnu services ganeti)?
I was "on the fence" about this myself, so I'm happy that you tipped me
over so to speak.
>> +Ganeti is a cluster-based virtual machine management system. It consists
> ^
> Maybe add one more sentence to give an idea of what it does or what
> features it provides.
That makes sense. I explained more in the transient blog post, but the
permanent documentation deserves better treatment.
>> +of multiple services which are described later in this section. In addition
>> +to the Ganeti service, you will need the OpenSSH service
>> +(@pxref{Networking Services, @code{openssh-service-type}}), and update the
>> +@file{/etc/hosts} file (@pxref{operating-system Reference, @code{hosts-file}})
>> +with the cluster name and address (or use a DNS server). Here is an example
>> +configuration for a Ganeti cluster node:
> ^
> Add “that does X and Y”, or “with X nodes running Y”, something like
> that. :-)
Thanks. :-)
>> +There is also a
>> +@url{https://guix.gnu.org/blog/2020/ganeti-cluster-on-guix/,blog post}
>> +describing how to configure a small cluster.
>
> It’d be great to see if part of the examples in the post (which I
> haven’t read yet) can be folded in the manual.
Agreed. I kind of wrote them together, before the service was
"finalized", and focused mostly on the end-to-end tutorial. I'll try to
extract generic parts into the manual.
>> +@table @asis
>> +@item @code{ganeti} (default: @code{ganeti})
>> +The @code{ganeti} package to use. It will be installed to the system profile
>> +and make @command{gnt-cluster}, @command{gnt-instance}, etc available. Note
>> +that the value specified here does not affect the other services as each refer
>> +to a specific @code{ganeti} package (see below).
>> +
>> +@item @code{noded-configuration} (default: @code{(ganeti-noded-configuration)})
>> +@item @code{confd-configuration} (default: @code{(ganeti-confd-configuration)})
>> +@item @code{wconfd-configuration} (default: @code{(ganeti-wconfd-configuration)})
>> +@item @code{luxid-configuration} (default: @code{(ganeti-luxid-configuration)})
>> +@item @code{rapi-configuration} (default: @code{(ganeti-rapi-configuration)})
>> +@item @code{kvmd-configuration} (default: @code{(ganeti-kvmd-configuration)})
>> +@item @code{mond-configuration} (default: @code{(ganeti-mond-configuration)})
>> +@item @code{watcher-configuration} (default: @code{(ganeti-watcher-configuration)})
>> +@item @code{cleaner-configuration} (default: @code{(ganeti-cleaner-configuration)})
>
> You need @itemx for all but the first one.
Ahh, that's much better, thanks again!
> Anyway, that looks very nice!
:-)
[signature.asc (application/pgp-signature, inline)]
guix-patches <at> gnu.org:bug#42261; Package guix-patches.
(Sat, 11 Jul 2020 22:09:02 GMT) Full text and rfc822 format available.Message #35 received at 42261 <at> debbugs.gnu.org (full text, mbox):
From: Marius Bakke <marius <at> gnu.org> To: Ludovic Courtès <ludo <at> gnu.org> Cc: 42261 <at> debbugs.gnu.org Subject: Re: [bug#42261] [PATCH 1/4] gnu: Add ganeti. Date: Sun, 12 Jul 2020 00:08:23 +0200
[Message part 1 (text/plain, inline)]
Ludovic Courtès <ludo <at> gnu.org> writes: > Marius Bakke <marius <at> gnu.org> skribis: > >> * gnu/packages/virtualization.scm (system->qemu-target, ganeti): New variables. >> * gnu/packages/patches/ganeti-copy-hmac.patch, >> gnu/packages/patches/ganeti-disable-version-symlinks.patch, >> gnu/packages/patches/ganeti-drbd-compat.patch, >> gnu/packages/patches/ganeti-haskell-pythondir.patch, >> gnu/packages/patches/ganeti-openvswitch-may-exist.patch, >> gnu/packages/patches/ganeti-preserve-PYTHONPATH.patch, >> gnu/packages/patches/ganeti-shepherd-master-failover.patch, >> gnu/packages/patches/ganeti-shepherd-support.patch: New files. >> * gnu/local.mk (dist_patch_DATA): Adjust accordingly. >> --- >> gnu/local.mk | 8 + >> gnu/packages/patches/ganeti-copy-hmac.patch | 83 ++++ >> .../ganeti-disable-version-symlinks.patch | 136 +++++++ >> gnu/packages/patches/ganeti-drbd-compat.patch | 168 ++++++++ >> .../patches/ganeti-haskell-pythondir.patch | 66 +++ >> .../ganeti-openvswitch-may-exist.patch | 25 ++ >> .../patches/ganeti-preserve-PYTHONPATH.patch | 21 + >> .../ganeti-shepherd-master-failover.patch | 26 ++ >> .../patches/ganeti-shepherd-support.patch | 87 ++++ >> gnu/packages/virtualization.scm | 378 ++++++++++++++++++ >> 10 files changed, 998 insertions(+) > > The patches are quite big, but maybe that’s unavoidable. Some have already been merged upstream, others should be shortly. Ultimately I hope to only carry the Shepherd- and PYTHONPATH-related patches. I haven't dared submitting 'disable-version-symlinks.patch' yet, mainly because I struggled to find a clean way to conditionally override Automake variables, ref <https://lists.gnu.org/archive/html/automake/2020-06/msg00000.html> (there is a reply from Karl Berry in 2020-07) > Apart from that, on a cursory look it LGTM! Awesome, I was able to do some more testing today and expect to push this in a couple of days.
[signature.asc (application/pgp-signature, inline)]
guix-patches <at> gnu.org:bug#42261; Package guix-patches.
(Mon, 13 Jul 2020 10:33:02 GMT) Full text and rfc822 format available.Message #38 received at 42261 <at> debbugs.gnu.org (full text, mbox):
From: Ludovic Courtès <ludo <at> gnu.org> To: Marius Bakke <marius <at> gnu.org> Cc: 42261 <at> debbugs.gnu.org Subject: Re: [bug#42261] [PATCH 1/4] gnu: Add ganeti. Date: Mon, 13 Jul 2020 12:32:18 +0200
Hi, Marius Bakke <marius <at> gnu.org> skribis: > Ludovic Courtès <ludo <at> gnu.org> writes: > >> Marius Bakke <marius <at> gnu.org> skribis: >> >>> * gnu/packages/virtualization.scm (system->qemu-target, ganeti): New variables. >>> * gnu/packages/patches/ganeti-copy-hmac.patch, >>> gnu/packages/patches/ganeti-disable-version-symlinks.patch, >>> gnu/packages/patches/ganeti-drbd-compat.patch, >>> gnu/packages/patches/ganeti-haskell-pythondir.patch, >>> gnu/packages/patches/ganeti-openvswitch-may-exist.patch, >>> gnu/packages/patches/ganeti-preserve-PYTHONPATH.patch, >>> gnu/packages/patches/ganeti-shepherd-master-failover.patch, >>> gnu/packages/patches/ganeti-shepherd-support.patch: New files. >>> * gnu/local.mk (dist_patch_DATA): Adjust accordingly. >>> --- >>> gnu/local.mk | 8 + >>> gnu/packages/patches/ganeti-copy-hmac.patch | 83 ++++ >>> .../ganeti-disable-version-symlinks.patch | 136 +++++++ >>> gnu/packages/patches/ganeti-drbd-compat.patch | 168 ++++++++ >>> .../patches/ganeti-haskell-pythondir.patch | 66 +++ >>> .../ganeti-openvswitch-may-exist.patch | 25 ++ >>> .../patches/ganeti-preserve-PYTHONPATH.patch | 21 + >>> .../ganeti-shepherd-master-failover.patch | 26 ++ >>> .../patches/ganeti-shepherd-support.patch | 87 ++++ >>> gnu/packages/virtualization.scm | 378 ++++++++++++++++++ >>> 10 files changed, 998 insertions(+) >> >> The patches are quite big, but maybe that’s unavoidable. > > Some have already been merged upstream, others should be shortly. > Ultimately I hope to only carry the Shepherd- and PYTHONPATH-related > patches. Great. > I haven't dared submitting 'disable-version-symlinks.patch' yet, > mainly because I struggled to find a clean way to conditionally > override Automake variables, ref > <https://lists.gnu.org/archive/html/automake/2020-06/msg00000.html> > > (there is a reply from Karl Berry in 2020-07) I think another option would be something like this (beware!): pkglibdir = if USE_VERSION_LINKS pkglibdir += foo else pkglibdir += bar endif >> Apart from that, on a cursory look it LGTM! > > Awesome, I was able to do some more testing today and expect to push > this in a couple of days. Yay! Ludo’.
Marius Bakke <marius <at> gnu.org>:Marius Bakke <marius <at> gnu.org>:Message #43 received at 42261-done <at> debbugs.gnu.org (full text, mbox):
From: Marius Bakke <marius <at> gnu.org> To: Ludovic Courtès <ludo <at> gnu.org> Cc: 42261-done <at> debbugs.gnu.org Subject: Re: [bug#42261] [PATCH 1/4] gnu: Add ganeti. Date: Thu, 16 Jul 2020 22:36:04 +0200
[Message part 1 (text/plain, inline)]
Ludovic Courtès <ludo <at> gnu.org> writes: >> I haven't dared submitting 'disable-version-symlinks.patch' yet, >> mainly because I struggled to find a clean way to conditionally >> override Automake variables, ref >> <https://lists.gnu.org/archive/html/automake/2020-06/msg00000.html> >> >> (there is a reply from Karl Berry in 2020-07) > > I think another option would be something like this (beware!): > > pkglibdir = > if USE_VERSION_LINKS > pkglibdir += foo > else > pkglibdir += bar > endif That's clever, but unfortunately did not work because the += adds an extra whitespace between entries. So in this case $(pkglibdir) would expand to ' bar', which may or may not work depending on whether the usage is quoted. For things like $(bindir), install-binSCRIPTS unfortunately quotes the entries. Tricky stuff! >>> Apart from that, on a cursory look it LGTM! >> >> Awesome, I was able to do some more testing today and expect to push >> this in a couple of days. > > Yay! I've made lots of tweaks recently, such as adding default hooks for debootstrap so that basic things work out of the box. Finally pushed now, and I intend to publish the blog post tomorrow. One step closer to worl^W datacenter domination. \o/
[signature.asc (application/pgp-signature, inline)]
Debbugs Internal Request <help-debbugs <at> gnu.org>
to internal_control <at> debbugs.gnu.org.
(Fri, 14 Aug 2020 11:24:06 GMT) Full text and rfc822 format available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.