GNU bug report logs - #40115
[PATCH] download: Use correct system and guile in 'url-fetch/tarbomb' and 'url-fetch/zipbomb'.

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 40115 in the body.
You can then email your comments to 40115 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Diego Nicola Barbato <dnbarbato <at> posteo.de>
To: guix-patches <at> gnu.org
Subject: [PATCH] download: Use correct system and guile in 'url-fetch/tarbomb'
 and 'url-fetch/zipbomb'.
Date: Wed, 18 Mar 2020 13:05:31 +0100

[Message part 1 (text/plain, inline)]

Hi Guix,

The attached patch fixes a bug where e.g.

  guix build -s i686-linux ffmpeg

builds a different derivation on i686-linux than on x86_64-linux.  This
doesn't just affect ffmpeg but a whole class of packages which use or
depend on a package that uses 'url-fetch/tarbomb' or 'url-fetch/zipbomb'
as the origin method of its source.  That's around 334 packages, among
them diffoscope, enlightenment, gnome, ungoogled-chromium, and wine.

The problem is fixed by explicitly passing the correct #:system and
#:guile-for-build to 'gexp->derivation' (as is done in other origin
methods such as 'git-fetch' or 'hg-fetch').

This shouldn't trigger any rebuils as it only affects the behaviour of
`guix build -s $system $package' if $system differs from the system type
of Guix itself.

Regards,

Diego

[0001-download-Use-correct-system-and-guile-in-url-fetch-t.patch (text/x-patch, attachment)]

Message #8 received at 40115 <at> debbugs.gnu.org (full text, mbox):

From: Diego Nicola Barbato <dnbarbato <at> posteo.de>
To: 40115 <at> debbugs.gnu.org
Subject: Re: [PATCH] download: Use correct system and guile in
 'url-fetch/tarbomb' and 'url-fetch/zipbomb'.
Date: Mon, 30 Mar 2020 22:11:57 +0200

[Message part 1 (text/plain, inline)]

Hey Guix,

Here's some additional information.

Diego Nicola Barbato <dnbarbato <at> posteo.de> writes:

> The attached patch fixes a bug where e.g.
>
>   guix build -s i686-linux ffmpeg
>
> builds a different derivation on i686-linux than on x86_64-linux.  This
> doesn't just affect ffmpeg but a whole class of packages which use or
> depend on a package that uses 'url-fetch/tarbomb' or 'url-fetch/zipbomb'
> as the origin method of its source.  That's around 334 packages, among
> them diffoscope, enlightenment, gnome, ungoogled-chromium, and wine.

The number (348 for commit 151f3d4) and full list of affected packages
can be computed by loading the attached script [0] into `guix repl' and
running `(show-affected-packages)'.

> The problem is fixed by explicitly passing the correct #:system and
> #:guile-for-build to 'gexp->derivation' (as is done in other origin
> methods such as 'git-fetch' or 'hg-fetch').
>
> This shouldn't trigger any rebuils as it only affects the behaviour of
> `guix build -s $system $package' if $system differs from the system type
> of Guix itself.

A closer look at some derivations and outputs suggests that this patch
will actually trigger rebuilds for all affected packages on all systems
except x86_64 because the build farm currently builds the wrong
derivations as can be seen for e.g. QEMU by comparing the build on
Cuirass

  https://ci.guix.gnu.org/build/2442001/details

with the derivations computed by

  guix build -s i686-linux --no-grafts -d qemu

on i686-linux and x86_64-linux (commit 151f3d4) respectively:

  Cuirass:
    /gnu/store/wc2k8h4iahbnfvl35220hvdx6mc70v7l-qemu-4.2.0.drv
    /gnu/store/fjg87f21qdzi7h5pqsxpd6rlf9mcy58h-qemu-4.2.0        <~
  i686-linux:
    /gnu/store/019ccjdh1nxfkpjyzwmirvif1ra9v3lh-qemu-4.2.0.drv
    /gnu/store/8a0cg5ip9967y54gkwskfxmiwwk9mf1b-qemu-4.2.0
  x86_64-linux:
    /gnu/store/iajzrw7lahcyhgyr7anmcjxa33607nqh-qemu-4.2.0.drv
    /gnu/store/fjg87f21qdzi7h5pqsxpd6rlf9mcy58h-qemu-4.2.0        <~

Consequently no substitutes are available for the affected packages on
systems other than x86_64-linux as witnessed by the different number of
available substitutes reported by

  guix weather -s i686-linux -m tarbomb-zipbomb-manifest-small.scm

on i686-linux

--8<---------------cut here---------------start------------->8---
computing 37 package derivations for i686-linux...
looking for 37 store items on https://ci.guix.gnu.org...
https://ci.guix.gnu.org
  18.9% substitutes available (7 out of 37)
  at least 2.3 MiB of nars (compressed)
  5.1 MiB on disk (uncompressed)
  0.001 seconds per request (0.0 seconds in total)
  1028.5 requests per second
  'https://ci.guix.gnu.org/api/queue?nr=1000' returned 504 ("Gateway Time-out")
--8<---------------cut here---------------end--------------->8---

and on x86_64-linux

--8<---------------cut here---------------start------------->8---
computing 37 package derivations for i686-linux...
looking for 37 store items on https://ci.guix.gnu.org...
https://ci.guix.gnu.org
  81.1% substitutes available (30 out of 37)
  at least 165.9 MiB of nars (compressed)
  423.3 MiB on disk (uncompressed)
  0.001 seconds per request (0.1 seconds in total)
  703.3 requests per second
  'https://ci.guix.gnu.org/api/queue?nr=1000' returned 504 ("Gateway Time-out")
--8<---------------cut here---------------end--------------->8---

I have attached manifest files for the packages directly using
`url-fetch/tarbomb' or `url-fetch/zipbomb' [1] and for all affected
packages [2] (they use the aforementioned script).

I think this patch can go on master even though it triggers more than
300 rebuilds, since there are currently no substitutes available for the
affected packages anyway.

Regards,

Diego

PS I hope I got all the terminology (e.g. computing vs. building a
derivation) right.

[0]: 

[uses-tarbomb-zipbomb.scm (application/octet-stream, attachment)]

[Message part 3 (text/plain, inline)]

[1]: 

[tarbomb-zipbomb-manifest-small.scm (application/octet-stream, attachment)]

[Message part 5 (text/plain, inline)]

[2]: 

[tarbomb-zipbomb-manifest-full.scm (application/octet-stream, attachment)]

Message #15 received at 40115-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Diego Nicola Barbato <dnbarbato <at> posteo.de>
Cc: 40115-done <at> debbugs.gnu.org
Subject: Re: [bug#40115] [PATCH] download: Use correct system and guile in
 'url-fetch/tarbomb' and 'url-fetch/zipbomb'.
Date: Wed, 08 Apr 2020 19:49:05 +0200

Hi Diego,

Diego Nicola Barbato <dnbarbato <at> posteo.de> skribis:

>>From 85594ce40c98ac5763b8295e2358567c6920188e Mon Sep 17 00:00:00 2001
> From: Diego Nicola Barbato <dnbarbato <at> posteo.de>
> Date: Mon, 16 Mar 2020 18:43:20 +0100
> Subject: [PATCH] download: Use correct system and guile in 'url-fetch/tarbomb'
>  and 'url-fetch/zipbomb'.
>
> Previously the result of `guix build -s $system $package' would depend on the
> system Guix was built for if $package or one of its dependencies used
> 'url-fetch/tarbomb' or 'url-fetch/zipbomb' as the origin method of its
> source (e.g. `guix build -s i686-linux ffmpeg' on i686-linux would build a
> different derivation than on x86_64-linux).
>
> This patch fixes this by explicitly passing the correct system and guile to
> 'gexp->derivation'.
>
> * guix/download.scm (url-fetch/tarbomb): Pass #:system system and
>   #:guile-for-build guile to 'gexp->derivation', where guile is the derivation
>   of guile for system.
>   (url-fetch/zipbomb): Likewise.

Good catch, pushed as c1d81df93d4b67671fc4a8e0a80c0f02c5821663!

>> builds a different derivation on i686-linux than on x86_64-linux.  This
>> doesn't just affect ffmpeg but a whole class of packages which use or
>> depend on a package that uses 'url-fetch/tarbomb' or 'url-fetch/zipbomb'
>> as the origin method of its source.  That's around 334 packages, among
>> them diffoscope, enlightenment, gnome, ungoogled-chromium, and wine.
>
> The number (348 for commit 151f3d4) and full list of affected packages
> can be computed by loading the attached script [0] into `guix repl' and
> running `(show-affected-packages)'.

Terrible that ci. has been building useless substitutes for these
packages and that users of non-x86_64 platforms were not getting
substitutes.

Thanks a lot for the patch and detailed analysis!

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.