GNU bug report logs - #43843
git-annex is not Reproducible

Previous Next

Package: guix;

Reported by: zimoun <zimon.toutoune <at> gmail.com>

Date: Wed, 7 Oct 2020 10:37:02 UTC

Severity: normal

Done: Marius Bakke <marius <at> gnu.org>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 43843 in the body.
You can then email your comments to 43843 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox

Report forwarded to bug-guix <at> gnu.org:
bug#43843; Package guix. (Wed, 07 Oct 2020 10:37:02 GMT) Full text and rfc822 format available.
Acknowledgement sent to zimoun <zimon.toutoune <at> gmail.com>:
New bug report received and forwarded. Copy sent to bug-guix <at> gnu.org. (Wed, 07 Oct 2020 10:37:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: zimoun <zimon.toutoune <at> gmail.com>
To: bug-guix <at> gnu.org
Subject: git-annex is not Reproducible
Date: Wed, 07 Oct 2020 12:35:51 +0200

Dear,

Using Guix 04a459a, the package ’git-annex’ is not reproducible:

  guix build git-annex
  guix build git-annex --no-grafts --check -K

return:

--8<---------------cut here---------------start------------->8---
guix build: error: derivation `/gnu/store/70qkv5hi5yq20m1h9jfm17m5lsc235mk-git-annex-8.20200908.drv' may not be deterministic: output `/gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908' differs from ‘/gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908-check’
--8<---------------cut here---------------end--------------->8---

then:

--8<---------------cut here---------------start------------->8---
$ diff -r --no-dereference /gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908{,-check}
Binary files /gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908/bin/git-annex and /gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908-check/bin/git-annex differ
--8<---------------cut here---------------end--------------->8---

Is it expected?  I have not tried ’diffoscope’ yet to investigate.


Note that:

<https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/git-annex.html>

speaks about GCC patch… I do not know if it is relevant.


All the best,
simon
Information forwarded to bug-guix <at> gnu.org:
bug#43843; Package guix. (Thu, 08 Oct 2020 03:05:02 GMT) Full text and rfc822 format available.

Message #8 received at 43843 <at> debbugs.gnu.org (full text, mbox):

From: Kyle Meyer <kyle <at> kyleam.com>
To: zimoun <zimon.toutoune <at> gmail.com>
Cc: 43843 <at> debbugs.gnu.org
Subject: Re: bug#43843: git-annex is not Reproducible
Date: Wed, 07 Oct 2020 23:04:13 -0400

On 10/07/20 12:35:51 +0200, zimoun wrote:
> Dear,
>
> Using Guix 04a459a, the package ’git-annex’ is not reproducible:
>
>   guix build git-annex
>   guix build git-annex --no-grafts --check -K
>
> return:
>
> --8<---------------cut here---------------start------------->8---
> guix build: error: derivation `/gnu/store/70qkv5hi5yq20m1h9jfm17m5lsc235mk-git-annex-8.20200908.drv' may not be deterministic: output `/gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908' differs from ‘/gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908-check’
> --8<---------------cut here---------------end--------------->8---
>
> then:
>
> --8<---------------cut here---------------start------------->8---
> $ diff -r --no-dereference /gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908{,-check}
> Binary files /gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908/bin/git-annex and /gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908-check/bin/git-annex differ
> --8<---------------cut here---------------end--------------->8---
>
> Is it expected?  I have not tried ’diffoscope’ yet to investigate.

Thanks for noticing.  I see the same on my end (and also with the
git-annex update [1] that I submitted before I saw this message).

I suspect this is a relatively new issue, as I've run `build --check'
successfully on git-annex in the past.  The git-annex definition prior
to the one above was introduced with e13fefbcff (gnu: git-annex: Create
fake static output, 2020-08-17).  On that commit, the check also failed
for me:

    $ guix time-machine --commit=e13fefbcffa50144d9301d67b61928ac3c964fb3 -- build --no-grafts --check git-annex
    guix build: error: derivation `/gnu/store/5illriaaijqrhl17aymzgiw6wzaq666s-git-annex-8.20200810.drv' may not be deterministic: output `/gnu/store/32h0xbqdwkm22naqba10fk9124cdgw9q-git-annex-8.20200810' differs

However, going back one more to 35e24fab73 (gnu: git-annex: Update to
8.20200810, 2020-08-11), the check was successful:

    $ guix time-machine --commit=35e24fab73bc450c6e0d3b9a5bbdfa88d2c425f4 -- build --no-grafts --check git-annex
    [...]
    /gnu/store/m79ri0g2s3d8pk52ysjkdvallk2zqbwb-git-annex-8.20200810


[1]: bug#43854: <20201007204946.22878-1-kyle <at> kyleam.com>
Information forwarded to bug-guix <at> gnu.org:
bug#43843; Package guix. (Thu, 08 Oct 2020 06:18:02 GMT) Full text and rfc822 format available.

Message #11 received at 43843 <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: Kyle Meyer <kyle <at> kyleam.com>
Cc: 43843 <at> debbugs.gnu.org, zimoun <zimon.toutoune <at> gmail.com>
Subject: Re: bug#43843: git-annex is not Reproducible
Date: Thu, 8 Oct 2020 09:17:13 +0300

[Message part 1 (text/plain, inline)]
On Wed, Oct 07, 2020 at 11:04:13PM -0400, Kyle Meyer wrote:
> On 10/07/20 12:35:51 +0200, zimoun wrote:
> > Dear,
> >
> > Using Guix 04a459a, the package ’git-annex’ is not reproducible:
> >
> >   guix build git-annex
> >   guix build git-annex --no-grafts --check -K
> >
> > return:
> >
> > --8<---------------cut here---------------start------------->8---
> > guix build: error: derivation `/gnu/store/70qkv5hi5yq20m1h9jfm17m5lsc235mk-git-annex-8.20200908.drv' may not be deterministic: output `/gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908' differs from ‘/gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908-check’
> > --8<---------------cut here---------------end--------------->8---
> >
> > then:
> >
> > --8<---------------cut here---------------start------------->8---
> > $ diff -r --no-dereference /gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908{,-check}
> > Binary files /gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908/bin/git-annex and /gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908-check/bin/git-annex differ
> > --8<---------------cut here---------------end--------------->8---
> >
> > Is it expected?  I have not tried ’diffoscope’ yet to investigate.
> 
> Thanks for noticing.  I see the same on my end (and also with the
> git-annex update [1] that I submitted before I saw this message).
> 
> I suspect this is a relatively new issue, as I've run `build --check'
> successfully on git-annex in the past.  The git-annex definition prior
> to the one above was introduced with e13fefbcff (gnu: git-annex: Create
> fake static output, 2020-08-17).  On that commit, the check also failed
> for me:
> 
>     $ guix time-machine --commit=e13fefbcffa50144d9301d67b61928ac3c964fb3 -- build --no-grafts --check git-annex
>     guix build: error: derivation `/gnu/store/5illriaaijqrhl17aymzgiw6wzaq666s-git-annex-8.20200810.drv' may not be deterministic: output `/gnu/store/32h0xbqdwkm22naqba10fk9124cdgw9q-git-annex-8.20200810' differs
> 
> However, going back one more to 35e24fab73 (gnu: git-annex: Update to
> 8.20200810, 2020-08-11), the check was successful:
> 
>     $ guix time-machine --commit=35e24fab73bc450c6e0d3b9a5bbdfa88d2c425f4 -- build --no-grafts --check git-annex
>     [...]
>     /gnu/store/m79ri0g2s3d8pk52ysjkdvallk2zqbwb-git-annex-8.20200810
> 
> 
> [1]: bug#43854: <20201007204946.22878-1-kyle <at> kyleam.com>
> 

In between those two commits is also the change to the haskell build
system update, where we use the dynamic libraries. The best bet is to
make sure the dependencies are built reproducibly.

-- 
Efraim Flashner   <efraim <at> flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[signature.asc (application/pgp-signature, inline)]
Information forwarded to bug-guix <at> gnu.org:
bug#43843; Package guix. (Fri, 09 Oct 2020 20:34:02 GMT) Full text and rfc822 format available.

Message #14 received at 43843 <at> debbugs.gnu.org (full text, mbox):

From: zimoun <zimon.toutoune <at> gmail.com>
To: Efraim Flashner <efraim <at> flashner.co.il>
Cc: rekado <at> elephly.net, Kyle Meyer <kyle <at> kyleam.com>, 43843 <at> debbugs.gnu.org
Subject: Re: bug#43843: git-annex is not Reproducible
Date: Fri, 09 Oct 2020 22:33:43 +0200

Dear,

On Thu, 08 Oct 2020 at 09:17, Efraim Flashner <efraim <at> flashner.co.il> wrote:

>>     $ guix time-machine --commit=e13fefbcffa50144d9301d67b61928ac3c964fb3 --
>> build --no-grafts --check git-annex
>>     guix build: error: derivation
>> `/gnu/store/5illriaaijqrhl17aymzgiw6wzaq666s-git-annex-8.20200810.drv' may
>> not be deterministic: output
>> `/gnu/store/32h0xbqdwkm22naqba10fk9124cdgw9q-git-annex-8.20200810' differs
>>
>> However, going back one more to 35e24fab73 (gnu: git-annex: Update to
>> 8.20200810, 2020-08-11), the check was successful:
>>
>>     $ guix time-machine --commit=35e24fab73bc450c6e0d3b9a5bbdfa88d2c425f4 --
>> build --no-grafts --check git-annex
>>     [...]
>>     /gnu/store/m79ri0g2s3d8pk52ysjkdvallk2zqbwb-git-annex-8.20200810

> In between those two commits is also the change to the haskell build
> system update, where we use the dynamic libraries. The best bet is to
> make sure the dependencies are built reproducibly.

Yep, it is what “git bisect” says.

--8<---------------cut here---------------start------------->8---
67cb9fa2357026ee42ec5bb0923ec4dc4a43abe2 is the first bad commit
commit 67cb9fa2357026ee42ec5bb0923ec4dc4a43abe2
Author: Ricardo Wurmus <rekado <at> elephly.net>
Date:   Tue Jun 16 22:25:48 2020 +0200

    build-system/haskell: Support parallel builds.

    * guix/build-system/haskell.scm (haskell-build): Add keyword
      PARALLEL-BUILD?
    and pass it on to the builder.
    * guix/build/haskell-build-system.scm (build): Accept keyword
      PARALLEL-BUILD?
    and pass the number of parallel jobs to GHC.

 guix/build-system/haskell.scm       | 2 ++
 guix/build/haskell-build-system.scm | 7 +++++--
 2 files changed, 7 insertions(+), 2 deletions(-)
bisect run success
--8<---------------cut here---------------end--------------->8---

I did:

  git bisect start e13fefbc 35e24fab
  git bisest run ./check.sh

where check.sh reads:

--8<---------------cut here---------------start------------->8---
#!/bin/sh

PKG=git-annex

git log -1 --format='%h %s [%an]'
COMMIT=$(git log -1 --format='%H')
echo "Start: $COMMIT"

guix time-machine     \
     --commit=$COMMIT \
     -- build $PKG    \
     --no-grafts

guix time-machine     \
     --commit=$COMMIT \
     -- build $PKG    \
     --no-grafts --check -K
status=$?

echo "End: [$status] $COMMIT"

exit $status
--8<---------------cut here---------------end--------------->8---


And I do not know if it is related, but these packages using the
’haskell-build-system’ are not reproducible neither (304/~640):

 agda
 cabal-install
 cpphs
 darcs
 dhall
 elm-compiler
 ghc-abstract-deque
 ghc-abstract-par
 ghc-adjunctions
 ghc-aeson
 ghc-alex
 ghc-asn1-encoding
 ghc-asn1-types
 ghc-assoc
 ghc-atomic-primops
 ghc-attoparsec
 ghc-auto-update
 ghc-aws
 ghc-base-compat
 ghc-basement
 ghc-base-unicode-symbols
 ghc-bifunctors
 ghc-bitarray
 ghc-blaze-html
 ghc-blaze-markup
 ghc-bloomfilter
 ghc-bytes
 ghc-bytestring-handle
 ghc-c2hs
 ghc-cairo
 ghc-cborg
 ghc-cereal
 ghc-cgi
 ghc-charset
 ghc-chart
 ghc-chasingbottoms
 ghc-cheapskate
 ghc-chunked-data
 ghc-clientsession
 ghc-cmdargs
 ghc-colour
 ghc-comonad
 ghc-concurrent-extra
 ghc-conduit
 ghc-conduit-algorithms
 ghc-conduit-extra
 ghc-configurator
 ghc-constraints
 ghc-convertible
 ghc-crypto-api
 ghc-crypto-api-tests
 ghc-crypto-cipher-tests
 ghc-crypto-cipher-types
 ghc-cryptohash
 ghc-cryptonite
 ghc-crypto-random
 ghc-curl
 ghc-data-accessor
 ghc-dbus
 ghc-descriptive
 ghc-diagrams-core
 ghc-diagrams-lib
 ghc-diagrams-solve
 ghc-diff
 ghc-digest
 ghc-doctest
 ghc-edisonapi
 ghc-edisoncore
 ghc-edit-distance
 ghc-either
 ghc-errors
 ghc-esqueleto
 ghc-exactprint
 ghc-extra
 ghc-fast-logger
 ghc-feed
 ghc-fgl
 ghc-filemanip
 ghc-filepath-bytestring
 ghc-fingertree
 ghc-foldl
 ghc-foundation
 ghc-free
 ghc-fsnotify
 ghc-generic-deriving
 ghc-generics-sop
 ghc-genvalidity-property
 ghc-getopt-generics
 ghc-gluraw
 ghc-glut
 ghc-gnuplot
 ghc-graphviz
 ghc-gtk2hs-buildtools
 ghc-hackage-security
 ghc-haddock-api
 ghc-haddock-library
 ghc-happstack-server
 ghc-happy
 ghc-hashtables
 ghc-haskeline
 ghc-haskell-lexer
 ghc-haskell-src
 ghc-haskell-src-exts
 ghc-haskell-src-exts-util
 ghc-hedgehog
 ghc-highlighting-kate
 ghc-hindent
 ghc-hmatrix
 ghc-hmatrix-gsl
 ghc-hmatrix-gsl-stats
 ghc-hmatrix-special
 ghc-hourglass
 ghc-hpack
 ghc-hslogger
 ghc-hslua
 ghc-hsopenssl
 ghc-hspec
 ghc-hspec-contrib
 ghc-hspec-core
 ghc-hspec-meta
 ghc-hsyaml
 ghc-http
 ghc-http2
 ghc-http-client
 ghc-http-date
 ghc-http-types
 ghc-hunit
 ghc-hxt
 ghc-hxt-regex-xmlschema
 ghc-hxt-unicode
 ghcid
 ghc-inline-c
 ghc-inspection-testing
 ghc-integer-logarithms
 ghc-interpolate
 ghc-intervalmap
 ghc-intervals
 ghc-io-streams
 ghc-jira-wiki-markup
 ghc-json
 ghc-juicypixels
 ghc-kan-extensions
 ghc-language-c
 ghc-language-glsl
 ghc-language-javascript
 ghc-lens
 ghc-lens-family-core
 ghc-libmpd
 ghc-lib-parser
 ghc-libxml
 ghc-libyaml
 ghc-lifted-base
 ghc-linear
 ghc-listlike
 ghc-llvm-hs
 ghc-llvm-hs-pure
 ghc-logging-facade
 ghc-math-functions
 ghc-megaparsec
 ghc-memory
 ghc-missingh
 ghc-monad-par
 ghc-monad-par-extras
 ghc-monads-tf
 ghc-monoid-extras
 ghc-mono-traversable
 ghc-mwc-random
 ghc-ncurses
 ghc-network
 ghc-non-negative
 ghc-opengl
 ghc-openglraw
 ghc-optparse-applicative
 ghc-pandoc
 ghc-pandoc-citeproc
 ghc-pandoc-types
 ghc-parallel
 ghc-parser-combinators
 ghc-parsers
 ghc-patience
 ghc-pem
 ghc-persistent
 ghc-persistent-test
 ghc-pgp-wordlist
 ghc-polyparse
 ghc-pqueue
 ghc-prettyprinter
 ghc-prettyprinter
 ghc-pretty-show
 ghc-pretty-simple
 ghc-primitive
 ghc-profunctors
 ghc-protolude
 ghc-psqueues
 ghc-quickcheck
 ghc-quickcheck-instances
 ghc-rebase
 ghc-reducers
 ghc-regex
 ghc-regex-applicative
 ghc-regex-pcre
 ghc-regex-pcre-builtin
 ghc-regex-posix
 ghc-regex-tdfa
 ghc-regex-tdfa-text
 ghc-rerebase
 ghc-rio
 ghc-safe
 ghc-safesemaphore
 ghc-sandi
 ghc-scientific
 ghc-sdl
 ghc-sdl2
 ghc-sdl2-image
 ghc-sdl-mixer
 ghc-semialign
 ghc-semigroupoids
 ghc-sendfile
 ghc-serialise
 ghc-shakespeare
 ghc-shelly
 ghc-size-based
 ghc-skylighting-core
 ghc-smallcheck
 ghc-snap-core
 ghc-snap-server
 ghc-socks
 ghc-sop-core
 ghc-sourcemap
 ghc-splitmix
 ghc-statistics
 ghc-stm-chans
 ghc-stm-conduit
 ghc-storable-record
 ghc-storablevector
 ghc-streaming-commons
 ghc-strict
 ghc-stringsearch
 ghc-stylish-haskell
 ghc-syb
 ghc-tagged
 ghc-tagsoup
 ghc-tar
 ghc-tasty
 ghc-test-framework
 ghc-testing-feat
 ghc-texmath
 ghc-tf-random
 ghc-these
 ghc-time-compat
 ghc-timezone-olson
 ghc-tls
 ghc-transformers
 ghc-transformers-compat
 ghc-tree-diff
 ghc-trifecta
 ghc-turtle
 ghc-unagi-chan
 ghc-unicode-transforms
 ghc-union-find
 ghc-uniplate
 ghc-unix-time
 ghc-unliftio
 ghc-unliftio-core
 ghc-unordered-containers
 ghc-uri-bytestring
 ghc-utf8-string
 ghc-utility-ht
 ghc-uuid
 ghc-vault
 ghc-vector
 ghc-vector-algorithms
 ghc-wai-extra
 ghc-warp
 ghc-websockets
 ghc-x11
 ghc-x509
 ghc-x509-store
 ghc-x509-system
 ghc-x509-validation
 ghc-xml
 ghc-xml-conduit
 ghc-xmonad-contrib
 ghc-yaml
 ghc-yesod
 ghc-yesod-core
 ghc-yesod-form
 ghc-zstd
 git-annex
 hlint
 hoogle
 hscolour
 hspec-discover
 idris
 kmonad
 ngless
 pandoc
 pandoc-citeproc
 purescript
 raincat
 scroll
 shellcheck
 stylish-haskell
 xmobar


All the best,
simon
Information forwarded to bug-guix <at> gnu.org:
bug#43843; Package guix. (Sat, 10 Oct 2020 09:35:01 GMT) Full text and rfc822 format available.

Message #17 received at 43843 <at> debbugs.gnu.org (full text, mbox):

From: Ricardo Wurmus <rekado <at> elephly.net>
To: zimoun <zimon.toutoune <at> gmail.com>
Cc: Kyle Meyer <kyle <at> kyleam.com>, Efraim Flashner <efraim <at> flashner.co.il>,
 43843 <at> debbugs.gnu.org
Subject: Re: bug#43843: git-annex is not Reproducible
Date: Sat, 10 Oct 2020 11:35:50 +0200

zimoun <zimon.toutoune <at> gmail.com> writes:

> Dear,
>
> On Thu, 08 Oct 2020 at 09:17, Efraim Flashner <efraim <at> flashner.co.il> wrote:
>
>>>     $ guix time-machine --commit=e13fefbcffa50144d9301d67b61928ac3c964fb3 --
>>> build --no-grafts --check git-annex
>>>     guix build: error: derivation
>>> `/gnu/store/5illriaaijqrhl17aymzgiw6wzaq666s-git-annex-8.20200810.drv' may
>>> not be deterministic: output
>>> `/gnu/store/32h0xbqdwkm22naqba10fk9124cdgw9q-git-annex-8.20200810' differs
>>>
>>> However, going back one more to 35e24fab73 (gnu: git-annex: Update to
>>> 8.20200810, 2020-08-11), the check was successful:
>>>
>>>     $ guix time-machine --commit=35e24fab73bc450c6e0d3b9a5bbdfa88d2c425f4 --
>>> build --no-grafts --check git-annex
>>>     [...]
>>>     /gnu/store/m79ri0g2s3d8pk52ysjkdvallk2zqbwb-git-annex-8.20200810
>
>> In between those two commits is also the change to the haskell build
>> system update, where we use the dynamic libraries. The best bet is to
>> make sure the dependencies are built reproducibly.
>
> Yep, it is what “git bisect” says.
>
> --8<---------------cut here---------------start------------->8---
> 67cb9fa2357026ee42ec5bb0923ec4dc4a43abe2 is the first bad commit
> commit 67cb9fa2357026ee42ec5bb0923ec4dc4a43abe2
> Author: Ricardo Wurmus <rekado <at> elephly.net>
> Date:   Tue Jun 16 22:25:48 2020 +0200
>
>     build-system/haskell: Support parallel builds.
>
>     * guix/build-system/haskell.scm (haskell-build): Add keyword
>       PARALLEL-BUILD?
>     and pass it on to the builder.
>     * guix/build/haskell-build-system.scm (build): Accept keyword
>       PARALLEL-BUILD?
>     and pass the number of parallel jobs to GHC.
>
>  guix/build-system/haskell.scm       | 2 ++
>  guix/build/haskell-build-system.scm | 7 +++++--
>  2 files changed, 7 insertions(+), 2 deletions(-)
> bisect run success
> --8<---------------cut here---------------end--------------->8---

Shall we revert it?  Previously the build system also supported parallel
builds, but didn’t seem to pass the right arguments to actually make it
happen.

-- 
Ricardo
Information forwarded to bug-guix <at> gnu.org:
bug#43843; Package guix. (Mon, 12 Oct 2020 09:32:02 GMT) Full text and rfc822 format available.

Message #20 received at 43843 <at> debbugs.gnu.org (full text, mbox):

From: zimoun <zimon.toutoune <at> gmail.com>
To: Ricardo Wurmus <rekado <at> elephly.net>
Cc: Kyle Meyer <kyle <at> kyleam.com>, Efraim Flashner <efraim <at> flashner.co.il>,
 43843 <at> debbugs.gnu.org
Subject: Re: bug#43843: git-annex is not Reproducible
Date: Mon, 12 Oct 2020 11:31:33 +0200

On Sat, 10 Oct 2020 at 11:34, Ricardo Wurmus <rekado <at> elephly.net> wrote:

> > --8<---------------cut here---------------start------------->8---
> > 67cb9fa2357026ee42ec5bb0923ec4dc4a43abe2 is the first bad commit
> > commit 67cb9fa2357026ee42ec5bb0923ec4dc4a43abe2
> > Author: Ricardo Wurmus <rekado <at> elephly.net>
> > Date:   Tue Jun 16 22:25:48 2020 +0200
> >
> >     build-system/haskell: Support parallel builds.
> >
> >     * guix/build-system/haskell.scm (haskell-build): Add keyword
> >       PARALLEL-BUILD?
> >     and pass it on to the builder.
> >     * guix/build/haskell-build-system.scm (build): Accept keyword
> >       PARALLEL-BUILD?
> >     and pass the number of parallel jobs to GHC.
> >
> >  guix/build-system/haskell.scm       | 2 ++
> >  guix/build/haskell-build-system.scm | 7 +++++--
> >  2 files changed, 7 insertions(+), 2 deletions(-)
> > bisect run success
> > --8<---------------cut here---------------end--------------->8---
>
> Shall we revert it?  Previously the build system also supported parallel
> builds, but didn’t seem to pass the right arguments to actually make it
> happen.

What do the Haskellers think about that?
Does the parallel build save a lot of time and CPU?  If yes, maybe we
could to provide a transformation for the expert, something like
"haskell-build-system-with-parellel-build" which tweaks
"PARALLEL-BUILD?", similarly to the recent "no tests".  WDYT?


All the best,
simon
Information forwarded to bug-guix <at> gnu.org:
bug#43843; Package guix. (Mon, 12 Oct 2020 10:35:02 GMT) Full text and rfc822 format available.

Message #23 received at 43843 <at> debbugs.gnu.org (full text, mbox):

From: Ricardo Wurmus <rekado <at> elephly.net>
To: zimoun <zimon.toutoune <at> gmail.com>
Cc: Kyle Meyer <kyle <at> kyleam.com>, Efraim Flashner <efraim <at> flashner.co.il>,
 43843 <at> debbugs.gnu.org
Subject: Re: bug#43843: git-annex is not Reproducible
Date: Mon, 12 Oct 2020 12:36:00 +0200

zimoun <zimon.toutoune <at> gmail.com> writes:

> On Sat, 10 Oct 2020 at 11:34, Ricardo Wurmus <rekado <at> elephly.net> wrote:
>
>> > --8<---------------cut here---------------start------------->8---
>> > 67cb9fa2357026ee42ec5bb0923ec4dc4a43abe2 is the first bad commit
>> > commit 67cb9fa2357026ee42ec5bb0923ec4dc4a43abe2
>> > Author: Ricardo Wurmus <rekado <at> elephly.net>
>> > Date:   Tue Jun 16 22:25:48 2020 +0200
>> >
>> >     build-system/haskell: Support parallel builds.
>> >
>> >     * guix/build-system/haskell.scm (haskell-build): Add keyword
>> >       PARALLEL-BUILD?
>> >     and pass it on to the builder.
>> >     * guix/build/haskell-build-system.scm (build): Accept keyword
>> >       PARALLEL-BUILD?
>> >     and pass the number of parallel jobs to GHC.
>> >
>> >  guix/build-system/haskell.scm       | 2 ++
>> >  guix/build/haskell-build-system.scm | 7 +++++--
>> >  2 files changed, 7 insertions(+), 2 deletions(-)
>> > bisect run success
>> > --8<---------------cut here---------------end--------------->8---
>>
>> Shall we revert it?  Previously the build system also supported parallel
>> builds, but didn’t seem to pass the right arguments to actually make it
>> happen.
>
> What do the Haskellers think about that?

Let me answer as an ex-Haskeller :)

> Does the parallel build save a lot of time and CPU?  If yes, maybe we
> could to provide a transformation for the expert, something like
> "haskell-build-system-with-parellel-build" which tweaks
> "PARALLEL-BUILD?", similarly to the recent "no tests".  WDYT?

We shouldn’t compromise reproducibility for parallel builds.  Ideally we
would figure out what exactly causes the differences and fix that
instead of disabling parallel builds, but if that turns out to be too
difficult I think we should just revert this until we have a good fix.

Perhaps something can be done by fixing the order of files somewhere.

-- 
Ricardo
Information forwarded to bug-guix <at> gnu.org:
bug#43843; Package guix. (Mon, 12 Oct 2020 11:57:02 GMT) Full text and rfc822 format available.

Message #26 received at 43843 <at> debbugs.gnu.org (full text, mbox):

From: zimoun <zimon.toutoune <at> gmail.com>
To: Ricardo Wurmus <rekado <at> elephly.net>
Cc: Kyle Meyer <kyle <at> kyleam.com>, Efraim Flashner <efraim <at> flashner.co.il>,
 43843 <at> debbugs.gnu.org
Subject: Re: bug#43843: git-annex is not Reproducible
Date: Mon, 12 Oct 2020 13:55:59 +0200

On Mon, 12 Oct 2020 at 12:34, Ricardo Wurmus <rekado <at> elephly.net> wrote:

> > Does the parallel build save a lot of time and CPU?  If yes, maybe we
> > could to provide a transformation for the expert, something like
> > "haskell-build-system-with-parellel-build" which tweaks
> > "PARALLEL-BUILD?", similarly to the recent "no tests".  WDYT?
>
> We shouldn’t compromise reproducibility for parallel builds.  Ideally we
> would figure out what exactly causes the differences and fix that
> instead of disabling parallel builds, but if that turns out to be too
> difficult I think we should just revert this until we have a good fix.
>
> Perhaps something can be done by fixing the order of files somewhere.

My proposal is for the expert and not at the CLI level.  Something
like "(funky-name ghc-foo)" which returns a new package with
PARALLEL-BUILD? turned to #t.  And the default should be #f.  WDYT?


Well, I am going to try to rebuild the packages with #f and see if it
fixes the ~300 unreproducible packages.


Cheers,
simon
Information forwarded to bug-guix <at> gnu.org:
bug#43843; Package guix. (Tue, 13 Oct 2020 13:07:02 GMT) Full text and rfc822 format available.

Message #29 received at 43843 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: zimoun <zimon.toutoune <at> gmail.com>
Cc: Ricardo Wurmus <rekado <at> elephly.net>, 43843 <at> debbugs.gnu.org
Subject: Re: bug#43843: git-annex is not Reproducible
Date: Tue, 13 Oct 2020 15:05:59 +0200

zimoun <zimon.toutoune <at> gmail.com> skribis:

> On Mon, 12 Oct 2020 at 12:34, Ricardo Wurmus <rekado <at> elephly.net> wrote:
>
>> > Does the parallel build save a lot of time and CPU?  If yes, maybe we
>> > could to provide a transformation for the expert, something like
>> > "haskell-build-system-with-parellel-build" which tweaks
>> > "PARALLEL-BUILD?", similarly to the recent "no tests".  WDYT?
>>
>> We shouldn’t compromise reproducibility for parallel builds.  Ideally we
>> would figure out what exactly causes the differences and fix that
>> instead of disabling parallel builds, but if that turns out to be too
>> difficult I think we should just revert this until we have a good fix.
>>
>> Perhaps something can be done by fixing the order of files somewhere.
>
> My proposal is for the expert and not at the CLI level.  Something
> like "(funky-name ghc-foo)" which returns a new package with
> PARALLEL-BUILD? turned to #t.  And the default should be #f.  WDYT?

You can always build with ‘--cores=1’, which will ensure both the build
and test will run sequentially.

Ludo’.
Information forwarded to bug-guix <at> gnu.org:
bug#43843; Package guix. (Tue, 13 Oct 2020 13:29:01 GMT) Full text and rfc822 format available.

Message #32 received at 43843 <at> debbugs.gnu.org (full text, mbox):

From: zimoun <zimon.toutoune <at> gmail.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Ricardo Wurmus <rekado <at> elephly.net>, 43843 <at> debbugs.gnu.org
Subject: Re: bug#43843: git-annex is not Reproducible
Date: Tue, 13 Oct 2020 15:28:02 +0200

On Tue, 13 Oct 2020 at 15:06, Ludovic Courtès <ludo <at> gnu.org> wrote:

> You can always build with ‘--cores=1’, which will ensure both the build
> and test will run sequentially.

Yes but in this case, "guix challenge" would still complain, isn't it?
Reply sent to Marius Bakke <marius <at> gnu.org>:
You have taken responsibility. (Sun, 18 Oct 2020 22:12:01 GMT) Full text and rfc822 format available.
Notification sent to zimoun <zimon.toutoune <at> gmail.com>:
bug acknowledged by developer. (Sun, 18 Oct 2020 22:12:01 GMT) Full text and rfc822 format available.

Message #37 received at 43843-done <at> debbugs.gnu.org (full text, mbox):

From: Marius Bakke <marius <at> gnu.org>
To: zimoun <zimon.toutoune <at> gmail.com>, 44049-done <at> debbugs.gnu.org
Cc: 43843-done <at> debbugs.gnu.org, zimoun <zimon.toutoune <at> gmail.com>
Subject: Re: [PATCH] build-system/haskell: Disable parallel builds.
Date: Mon, 19 Oct 2020 00:11:30 +0200

[Message part 1 (text/plain, inline)]
zimoun <zimon.toutoune <at> gmail.com> writes:

> Fixes <https://bugs.gnu.org/43843>.
>
> PARALLEL-BUILD? introduced by commit 67cb9fa2357026ee42ec5bb0923ec4dc4a43abe2
> leads to unreproducibility.  Instead of reverting, default is set to #false
> which allows user to locally build with parallelism.
>
> * guix/build-system/haskell.scm (haskell-build): Turn off PARALLEL-BUILD? by
> default.

Thanks!  I shortened the comment a little and pushed to staging in
19d42e0e23a7f90ac2dcc1c279bd23a967ff0314.

[signature.asc (application/pgp-signature, inline)]
bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Mon, 16 Nov 2020 12:24:04 GMT) Full text and rfc822 format available.
This bug report was last modified 3 years and 155 days ago.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.