GNU bug report logs -
#27593
groff build is not reproducible
Previous Next
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 27593 in the body.
You can then email your comments to 27593 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
bug-guix <at> gnu.org
:
bug#27593
; Package
guix
.
(Thu, 06 Jul 2017 00:56:01 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
Danny Milosavljevic <dannym <at> scratchpost.org>
:
New bug report received and forwarded. Copy sent to
bug-guix <at> gnu.org
.
(Thu, 06 Jul 2017 00:56:02 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
on core-updates
Information forwarded
to
bug-guix <at> gnu.org
:
bug#27593
; Package
guix
.
(Thu, 06 Jul 2017 01:02:02 GMT)
Full text and
rfc822 format available.
Message #8 received at 27593 <at> debbugs.gnu.org (full text, mbox):
Fixes <https://bugs.gnu.org/27593>.
* gnu/packages/groff.scm (groff)[source]: Remove timestamps.
---
gnu/packages/groff.scm | 15 ++++++++++++++-
1 file changed, 14 insertions(+), 1 deletion(-)
diff --git a/gnu/packages/groff.scm b/gnu/packages/groff.scm
index 46e1ccf23..e5dd732ab 100644
--- a/gnu/packages/groff.scm
+++ b/gnu/packages/groff.scm
@@ -40,7 +40,20 @@
(uri (string-append "mirror://gnu/groff/groff-" version
".tar.gz"))
(sha256 (base32
- "1998v2kcs288d3y7kfxpvl369nqi06zbbvjzafyvyl3pr7bajj1s"))))
+ "1998v2kcs288d3y7kfxpvl369nqi06zbbvjzafyvyl3pr7bajj1s"))
+ (modules '((guix build utils)))
+ (snippet
+ '(begin
+ (substitute* "mdate.sh"
+ (("^LANGUAGE=") "
+
+echo \"1 Jan 1970\"
+exit 0
+
+"))
+ (substitute* '("src/devices/grops/ps.cpp"
+ "src/devices/grohtml/post-html.cpp")
+ (("time\\(0\\)") "0"))))))
(build-system gnu-build-system)
(outputs '("out"
"doc")) ;12MiB of PS, PDF, HTML, and examples
Information forwarded
to
bug-guix <at> gnu.org
:
bug#27593
; Package
guix
.
(Thu, 06 Jul 2017 02:04:01 GMT)
Full text and
rfc822 format available.
Message #11 received at 27593 <at> debbugs.gnu.org (full text, mbox):
Fixes <https://bugs.gnu.org/27593>.
* gnu/packages/groff.scm (groff)[source]: Remove timestamps.
---
gnu/packages/groff.scm | 18 +++++++++++++++++-
1 file changed, 17 insertions(+), 1 deletion(-)
diff --git a/gnu/packages/groff.scm b/gnu/packages/groff.scm
index 46e1ccf23..1a74194a6 100644
--- a/gnu/packages/groff.scm
+++ b/gnu/packages/groff.scm
@@ -40,7 +40,23 @@
(uri (string-append "mirror://gnu/groff/groff-" version
".tar.gz"))
(sha256 (base32
- "1998v2kcs288d3y7kfxpvl369nqi06zbbvjzafyvyl3pr7bajj1s"))))
+ "1998v2kcs288d3y7kfxpvl369nqi06zbbvjzafyvyl3pr7bajj1s"))
+ (modules '((guix build utils)))
+ (snippet
+ '(begin
+ (substitute* "mdate.sh"
+ (("^LANGUAGE=") "
+
+echo \"1 Jan 1970\"
+exit 0
+
+"))
+ (substitute* '("src/devices/grops/ps.cpp"
+ "src/devices/grohtml/post-html.cpp"
+ "src/roff/troff/input.cpp")
+ (("time\\(0\\)") "0"))
+ (substitute* "src/devices/gropdf/gropdf.pl"
+ (("\\(time\\)") "(0)"))))))
(build-system gnu-build-system)
(outputs '("out"
"doc")) ;12MiB of PS, PDF, HTML, and examples
Information forwarded
to
bug-guix <at> gnu.org
:
bug#27593
; Package
guix
.
(Sun, 09 Jul 2017 13:01:01 GMT)
Full text and
rfc822 format available.
Message #14 received at 27593 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Danny Milosavljevic <dannym <at> scratchpost.org> writes:
> Fixes <https://bugs.gnu.org/27593>.
>
> * gnu/packages/groff.scm (groff)[source]: Remove timestamps.
> ---
> gnu/packages/groff.scm | 18 +++++++++++++++++-
> 1 file changed, 17 insertions(+), 1 deletion(-)
>
> diff --git a/gnu/packages/groff.scm b/gnu/packages/groff.scm
> index 46e1ccf23..1a74194a6 100644
> --- a/gnu/packages/groff.scm
> +++ b/gnu/packages/groff.scm
> @@ -40,7 +40,23 @@
> (uri (string-append "mirror://gnu/groff/groff-" version
> ".tar.gz"))
> (sha256 (base32
> - "1998v2kcs288d3y7kfxpvl369nqi06zbbvjzafyvyl3pr7bajj1s"))))
> + "1998v2kcs288d3y7kfxpvl369nqi06zbbvjzafyvyl3pr7bajj1s"))
> + (modules '((guix build utils)))
> + (snippet
> + '(begin
> + (substitute* "mdate.sh"
> + (("^LANGUAGE=") "
> +
> +echo \"1 Jan 1970\"
> +exit 0
> +
> +"))
It would be more obvious what's going on if we substitute the "mdate.sh"
invocation from the Makefile instead. OTOH, this is more
future-proof. No strong opinion, but feels a bit "hacky".
Note that you can use \n here.
> + (substitute* '("src/devices/grops/ps.cpp"
> + "src/devices/grohtml/post-html.cpp"
> + "src/roff/troff/input.cpp")
> + (("time\\(0\\)") "0"))
> + (substitute* "src/devices/gropdf/gropdf.pl"
> + (("\\(time\\)") "(0)"))))))
Are all of these really a cause of indeterminism in the build process?
It looks like this could make "legitimate" uses of groff return epoch 0
at runtime, I'm not sure if that is okay.
Which output files are fixed by these substitutions? Are there other
ways to deal with the indeterminism, e.g. substitute them in place?
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
bug-guix <at> gnu.org
:
bug#27593
; Package
guix
.
(Sun, 09 Jul 2017 19:52:01 GMT)
Full text and
rfc822 format available.
Message #17 received at 27593 <at> debbugs.gnu.org (full text, mbox):
Marius Bakke <mbakke <at> fastmail.com> skribis:
> Danny Milosavljevic <dannym <at> scratchpost.org> writes:
>
>> Fixes <https://bugs.gnu.org/27593>.
>>
>> * gnu/packages/groff.scm (groff)[source]: Remove timestamps.
>> ---
>> gnu/packages/groff.scm | 18 +++++++++++++++++-
>> 1 file changed, 17 insertions(+), 1 deletion(-)
>>
>> diff --git a/gnu/packages/groff.scm b/gnu/packages/groff.scm
>> index 46e1ccf23..1a74194a6 100644
>> --- a/gnu/packages/groff.scm
>> +++ b/gnu/packages/groff.scm
>> @@ -40,7 +40,23 @@
>> (uri (string-append "mirror://gnu/groff/groff-" version
>> ".tar.gz"))
>> (sha256 (base32
>> - "1998v2kcs288d3y7kfxpvl369nqi06zbbvjzafyvyl3pr7bajj1s"))))
>> + "1998v2kcs288d3y7kfxpvl369nqi06zbbvjzafyvyl3pr7bajj1s"))
>> + (modules '((guix build utils)))
>> + (snippet
>> + '(begin
>> + (substitute* "mdate.sh"
>> + (("^LANGUAGE=") "
>> +
>> +echo \"1 Jan 1970\"
>> +exit 0
>> +
>> +"))
>
> It would be more obvious what's going on if we substitute the "mdate.sh"
> invocation from the Makefile instead. OTOH, this is more
> future-proof. No strong opinion, but feels a bit "hacky".
>
> Note that you can use \n here.
>
>> + (substitute* '("src/devices/grops/ps.cpp"
>> + "src/devices/grohtml/post-html.cpp"
>> + "src/roff/troff/input.cpp")
>> + (("time\\(0\\)") "0"))
>> + (substitute* "src/devices/gropdf/gropdf.pl"
>> + (("\\(time\\)") "(0)"))))))
>
> Are all of these really a cause of indeterminism in the build process?
> It looks like this could make "legitimate" uses of groff return epoch 0
> at runtime, I'm not sure if that is okay.
>
> Which output files are fixed by these substitutions? Are there other
> ways to deal with the indeterminism, e.g. substitute them in place?
Sometimes it’s better to make such changes available as a .patch file so
that we can more clearly see from the context that we’re patching the
right thing. Perhaps that’s one such case here.
BTW, looking at issues.yml in
git://git.debian.org/git/reproducible/notes.git leads me to:
https://wiki.debian.org/ReproducibleBuilds/TimestampsInDocumentationGeneratedByGroff
Debian has a SOURCE_DATE_EPOCH patch:
http://sources.debian.net/patches/groff/1.22.3-9/source-date-epoch.patch/
Perhaps we could just use that? That way, the default behavior of groff
would remained unchanged.
Apparently upstream wasn’t convinced by the idea of SOURCE_DATE_EPOCH,
so we could also restart the discussion with upstream in parallel.
However, it would be nice to commit this groff change quickly so we can
go on with core-updates builds. :-)
Thoughts?
Ludo’.
Information forwarded
to
bug-guix <at> gnu.org
:
bug#27593
; Package
guix
.
(Sun, 09 Jul 2017 21:28:01 GMT)
Full text and
rfc822 format available.
Message #20 received at 27593 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Hi Ludo,
On Sun, 09 Jul 2017 21:51:08 +0200
ludo <at> gnu.org (Ludovic Courtès) wrote:
> Debian has a SOURCE_DATE_EPOCH patch:
>
> http://sources.debian.net/patches/groff/1.22.3-9/source-date-epoch.patch/
>
> Perhaps we could just use that? That way, the default behavior of groff
> would remained unchanged.
Yeah, it seems to work fine.
Apparently git send-email doesn't work for me right now.
So I attached the patch manually here...
[0001-gnu-groff-Remove-timestamps.patch (text/x-patch, attachment)]
Information forwarded
to
bug-guix <at> gnu.org
:
bug#27593
; Package
guix
.
(Sun, 09 Jul 2017 21:56:01 GMT)
Full text and
rfc822 format available.
Message #23 received at 27593 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Danny Milosavljevic <dannym <at> scratchpost.org> writes:
> Hi Ludo,
>
> On Sun, 09 Jul 2017 21:51:08 +0200
> ludo <at> gnu.org (Ludovic Courtès) wrote:
>> Debian has a SOURCE_DATE_EPOCH patch:
>>
>> http://sources.debian.net/patches/groff/1.22.3-9/source-date-epoch.patch/
>>
>> Perhaps we could just use that? That way, the default behavior of groff
>> would remained unchanged.
>
> Yeah, it seems to work fine.
>
> Apparently git send-email doesn't work for me right now.
>
> So I attached the patch manually here...
[...]
> - "1998v2kcs288d3y7kfxpvl369nqi06zbbvjzafyvyl3pr7bajj1s"))))
> + "1998v2kcs288d3y7kfxpvl369nqi06zbbvjzafyvyl3pr7bajj1s"))
> + (patches (search-patches "groff-source-date-epoch.patch"))
> + (modules '((guix build utils)))))
(guix build utils) is no longer needed. Apart from that LGTM!
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
bug-guix <at> gnu.org
:
bug#27593
; Package
guix
.
(Mon, 10 Jul 2017 09:07:02 GMT)
Full text and
rfc822 format available.
Message #26 received at 27593 <at> debbugs.gnu.org (full text, mbox):
Marius Bakke <mbakke <at> fastmail.com> skribis:
> Danny Milosavljevic <dannym <at> scratchpost.org> writes:
>
>> Hi Ludo,
>>
>> On Sun, 09 Jul 2017 21:51:08 +0200
>> ludo <at> gnu.org (Ludovic Courtès) wrote:
>>> Debian has a SOURCE_DATE_EPOCH patch:
>>>
>>> http://sources.debian.net/patches/groff/1.22.3-9/source-date-epoch.patch/
>>>
>>> Perhaps we could just use that? That way, the default behavior of groff
>>> would remained unchanged.
>>
>> Yeah, it seems to work fine.
>>
>> Apparently git send-email doesn't work for me right now.
>>
>> So I attached the patch manually here...
>
> [...]
>
>> - "1998v2kcs288d3y7kfxpvl369nqi06zbbvjzafyvyl3pr7bajj1s"))))
>> + "1998v2kcs288d3y7kfxpvl369nqi06zbbvjzafyvyl3pr7bajj1s"))
>> + (patches (search-patches "groff-source-date-epoch.patch"))
>> + (modules '((guix build utils)))))
>
> (guix build utils) is no longer needed. Apart from that LGTM!
+1!
Ludo'.
Information forwarded
to
bug-guix <at> gnu.org
:
bug#27593
; Package
guix
.
(Mon, 10 Jul 2017 10:41:01 GMT)
Full text and
rfc822 format available.
Message #29 received at 27593 <at> debbugs.gnu.org (full text, mbox):
Thanks!
Pushed to core-updates as 9146543f8dce997c194aea93bf4c7a7a4144c8c9. Note that for real reproducibility the ghostscript patches are required too, and a phase in groff that sets the environment variable.
I've sent a new series for ghostscript.
Information forwarded
to
bug-guix <at> gnu.org
:
bug#27593
; Package
guix
.
(Mon, 10 Jul 2017 16:38:02 GMT)
Full text and
rfc822 format available.
Message #32 received at 27593 <at> debbugs.gnu.org (full text, mbox):
* gnu/packages/groff.scm (groff)[arguments]: Add phase "setenv".
---
gnu/packages/groff.scm | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/gnu/packages/groff.scm b/gnu/packages/groff.scm
index 67dd1dbfa..6e1a60e9b 100644
--- a/gnu/packages/groff.scm
+++ b/gnu/packages/groff.scm
@@ -56,7 +56,14 @@
("perl" ,perl)
("psutils" ,psutils)
("texinfo" ,texinfo)))
- (arguments '(#:parallel-build? #f)) ; parallel build fails
+ (arguments
+ `(#:parallel-build? #f ; parallel build fails
+ #:phases
+ (modify-phases %standard-phases
+ (add-after 'unpack 'setenv
+ (lambda _
+ (setenv "GS_GENERATE_UUIDS" "0")
+ #t)))))
(synopsis "Typesetting from plain text mixed with formatting commands")
(description
"Groff is a typesetting package that reads plain text and produces
Information forwarded
to
bug-guix <at> gnu.org
:
bug#27593
; Package
guix
.
(Mon, 10 Jul 2017 19:08:02 GMT)
Full text and
rfc822 format available.
Message #35 received at 27593 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Danny Milosavljevic <dannym <at> scratchpost.org> writes:
> * gnu/packages/groff.scm (groff)[arguments]: Add phase "setenv".
What does this add to the previous patch? Can you add a comment in the
code explaining what it's for?
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
bug-guix <at> gnu.org
:
bug#27593
; Package
guix
.
(Mon, 10 Jul 2017 19:29:02 GMT)
Full text and
rfc822 format available.
Message #38 received at 27593 <at> debbugs.gnu.org (full text, mbox):
Hi Marius,
On Mon, 10 Jul 2017 21:07:00 +0200
Marius Bakke <mbakke <at> fastmail.com> wrote:
> Danny Milosavljevic <dannym <at> scratchpost.org> writes:
>
> > * gnu/packages/groff.scm (groff)[arguments]: Add phase "setenv".
>
> What does this add to the previous patch? Can you add a comment in the
> code explaining what it's for?
It replaces the previous patch by using a new patched-ghostscript feature:
- ghostscript can now be instructed to not print the problematic parts (timestamps in headers, UUIDs in headers) that were the original cause of the "ever-changing PDFs" problem.
The previous approach was fighting the symptoms, where every time a problem appeared in one of the outputs, I'd patch it somewhere else so that, if it did make it to ghostscript, ghostscript would put fake timestamps and would derive UUIDs from them - which upstream recommended we should definitely not do.
That was really not scalable or sane.
Also, the original ghostscript states that they are printing the current system date into PDFs (rather than, say, the input file mtime), against better judgement, for compatibility. Just no.
So this patch prevents ghostscript from putting the timestamps or the document UUIDs into the PDFs in the first place (I don't mean it puts a fake one there now - I mean it leaves the fields off entirely). Much simpler.
I'll add a comment.
Information forwarded
to
bug-guix <at> gnu.org
:
bug#27593
; Package
guix
.
(Tue, 11 Jul 2017 09:32:01 GMT)
Full text and
rfc822 format available.
Message #41 received at 27593 <at> debbugs.gnu.org (full text, mbox):
Danny Milosavljevic <dannym <at> scratchpost.org> skribis:
> * gnu/packages/groff.scm (groff)[arguments]: Add phase "setenv".
> ---
> gnu/packages/groff.scm | 9 ++++++++-
> 1 file changed, 8 insertions(+), 1 deletion(-)
>
> diff --git a/gnu/packages/groff.scm b/gnu/packages/groff.scm
> index 67dd1dbfa..6e1a60e9b 100644
> --- a/gnu/packages/groff.scm
> +++ b/gnu/packages/groff.scm
> @@ -56,7 +56,14 @@
> ("perl" ,perl)
> ("psutils" ,psutils)
> ("texinfo" ,texinfo)))
> - (arguments '(#:parallel-build? #f)) ; parallel build fails
> + (arguments
> + `(#:parallel-build? #f ; parallel build fails
> + #:phases
> + (modify-phases %standard-phases
> + (add-after 'unpack 'setenv
> + (lambda _
> + (setenv "GS_GENERATE_UUIDS" "0")
> + #t)))))
With a comment like Marius wrote, it’d be perfect; otherwise LGTM.
However, since ‘core-updates’ is now building and we’d like to make
progress on it, could you push it to ‘core-updates-next’ or similar?
Thanks,
Ludo’.
Information forwarded
to
bug-guix <at> gnu.org
:
bug#27593
; Package
guix
.
(Sat, 09 Sep 2017 20:32:02 GMT)
Full text and
rfc822 format available.
Message #44 received at 27593 <at> debbugs.gnu.org (full text, mbox):
Hi Danny,
ludo <at> gnu.org (Ludovic Courtès) skribis:
> Danny Milosavljevic <dannym <at> scratchpost.org> skribis:
>
>> * gnu/packages/groff.scm (groff)[arguments]: Add phase "setenv".
>> ---
>> gnu/packages/groff.scm | 9 ++++++++-
>> 1 file changed, 8 insertions(+), 1 deletion(-)
>>
>> diff --git a/gnu/packages/groff.scm b/gnu/packages/groff.scm
>> index 67dd1dbfa..6e1a60e9b 100644
>> --- a/gnu/packages/groff.scm
>> +++ b/gnu/packages/groff.scm
>> @@ -56,7 +56,14 @@
>> ("perl" ,perl)
>> ("psutils" ,psutils)
>> ("texinfo" ,texinfo)))
>> - (arguments '(#:parallel-build? #f)) ; parallel build fails
>> + (arguments
>> + `(#:parallel-build? #f ; parallel build fails
>> + #:phases
>> + (modify-phases %standard-phases
>> + (add-after 'unpack 'setenv
>> + (lambda _
>> + (setenv "GS_GENERATE_UUIDS" "0")
>> + #t)))))
>
> With a comment like Marius wrote, it’d be perfect; otherwise LGTM.
>
> However, since ‘core-updates’ is now building and we’d like to make
> progress on it, could you push it to ‘core-updates-next’ or similar?
Maybe now is the time to push this to ‘core-updates’?
(I was just hit by this reproducibility issue for a PDF file made with
groff…)
Cheers,
Ludo’.
Information forwarded
to
bug-guix <at> gnu.org
:
bug#27593
; Package
guix
.
(Sat, 09 Sep 2017 21:18:02 GMT)
Full text and
rfc822 format available.
Message #47 received at 27593 <at> debbugs.gnu.org (full text, mbox):
Hi Ludo,
On Sat, 09 Sep 2017 22:31:36 +0200
ludo <at> gnu.org (Ludovic Courtès) wrote:
> Maybe now is the time to push this to ‘core-updates’?
>
> (I was just hit by this reproducibility issue for a PDF file made with
> groff…)
Isn't it pushed already?
If so, bad sign. Maybe the new environment variable doesn't always work?
Do you have a diff of what is changed in the PDF? It should have left the creationdate and moddate off, so it shouldn't list those in the diff...
Reply sent
to
ludo <at> gnu.org (Ludovic Courtès)
:
You have taken responsibility.
(Sun, 10 Sep 2017 13:14:01 GMT)
Full text and
rfc822 format available.
Notification sent
to
Danny Milosavljevic <dannym <at> scratchpost.org>
:
bug acknowledged by developer.
(Sun, 10 Sep 2017 13:14:02 GMT)
Full text and
rfc822 format available.
Message #52 received at 27593-done <at> debbugs.gnu.org (full text, mbox):
Hi Danny,
Danny Milosavljevic <dannym <at> scratchpost.org> skribis:
> Hi Ludo,
>
> On Sat, 09 Sep 2017 22:31:36 +0200
> ludo <at> gnu.org (Ludovic Courtès) wrote:
>
>> Maybe now is the time to push this to ‘core-updates’?
>>
>> (I was just hit by this reproducibility issue for a PDF file made with
>> groff…)
>
> Isn't it pushed already?
Oh it is, but in the new ‘core-updates’ (I was looking at ‘master’.).
Perfect, happily closing the bug now. :-)
Thanks,
Ludo’.
bug archived.
Request was from
Debbugs Internal Request <help-debbugs <at> gnu.org>
to
internal_control <at> debbugs.gnu.org
.
(Mon, 09 Oct 2017 11:24:05 GMT)
Full text and
rfc822 format available.
This bug report was last modified 6 years and 194 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.