GNU bug report logs - #27563
ghostscript-with-cups is not reproducible

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 27563 in the body.
You can then email your comments to 27563 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: <bug-guix <at> gnu.org>
Subject: ghostscript-with-cups is not reproducible
Date: Mon, 3 Jul 2017 20:08:44 +0200

diff -ur --no-dereference guix-build-ghostscript-with-cups-9.14.0.drv-0/gnu-ghostscript-9.14.0/soobj/gsromfs1_.c guix-build-ghostscript-with-cups-9.14.0.drv-1/gnu-ghostscript-9.14.0/
soobj/gsromfs1_.c
--- guix-build-ghostscript-with-cups-9.14.0.drv-0/gnu-ghostscript-9.14.0/soobj/gsromfs1_.c      2017-07-03 19:45:46.632983314 +0200
+++ guix-build-ghostscript-with-cups-9.14.0.drv-1/gnu-ghostscript-9.14.0/soobj/gsromfs1_.c      2017-07-03 19:56:25.401286255 +0200
@@ -5,7 +5,7 @@
 
 #include "time_.h"
 
-    time_t gs_romfs_buildtime = 1499103945;
+    time_t gs_romfs_buildtime = 1499104584;
 
     static uint32_t node_0[] = {
        0x5cdc0280,     /* compression_flag_bit + file length */
diff -ur --no-dereference guix-build-ghostscript-with-cups-9.14.0.drv-0/gnu-ghostscript-9.14.0/soobj/gsromfs1.c guix-build-ghostscript-with-cups-9.14.0.drv-1/gnu-ghostscript-9.14.0/soobj/gsromfs1.c
--- guix-build-ghostscript-with-cups-9.14.0.drv-0/gnu-ghostscript-9.14.0/soobj/gsromfs1.c       2017-07-03 19:45:46.660983573 +0200
+++ guix-build-ghostscript-with-cups-9.14.0.drv-1/gnu-ghostscript-9.14.0/soobj/gsromfs1.c       2017-07-03 19:56:25.437286605 +0200
@@ -5,7 +5,7 @@
 
 #include "time_.h"
 
-    time_t gs_romfs_buildtime = 1499103945;
+    time_t gs_romfs_buildtime = 1499104584;
 
     static uint32_t node_0[] = {
        0x5cdc0280,     /* compression_flag_bit + file length */

Message #8 received at 27563 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: 27563 <at> debbugs.gnu.org
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>
Subject: [PATCH] gnu: ghostscript: Add phase "remove-timestamps".
Date: Mon,  3 Jul 2017 20:16:46 +0200

* gnu/packages/ghostscript.scm (ghostscript): Add phase "remove-timestamps".
---
 gnu/packages/ghostscript.scm | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/gnu/packages/ghostscript.scm b/gnu/packages/ghostscript.scm
index 1cb651c96..56e05cd77 100644
--- a/gnu/packages/ghostscript.scm
+++ b/gnu/packages/ghostscript.scm
@@ -185,6 +185,11 @@ printing, and psresize, for adjusting page sizes.")
                      (substitute* "base/gscdef.c"
                        (("GS_DOCDIR")
                         "\"~/.guix-profile/share/doc/ghostscript\""))))
+        (add-after 'configure 'remove-timestamps
+                   (lambda _
+                     (substitute* "base/mkromfs.c"
+                       (("time\\(NULL\\)") "0U"))
+                     #t))
         (replace 'build
           (lambda _
             ;; Build 'libgs.so', but don't build the statically-linked 'gs'

Message #11 received at 27563 <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 27563 <at> debbugs.gnu.org
Subject: Re: bug#27563: [PATCH] gnu: ghostscript: Add phase
 "remove-timestamps".
Date: Mon, 3 Jul 2017 22:18:58 +0300

[Message part 1 (text/plain, inline)]

On Mon, Jul 03, 2017 at 08:16:46PM +0200, Danny Milosavljevic wrote:
> * gnu/packages/ghostscript.scm (ghostscript): Add phase "remove-timestamps".
> ---
>  gnu/packages/ghostscript.scm | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/gnu/packages/ghostscript.scm b/gnu/packages/ghostscript.scm
> index 1cb651c96..56e05cd77 100644
> --- a/gnu/packages/ghostscript.scm
> +++ b/gnu/packages/ghostscript.scm
> @@ -185,6 +185,11 @@ printing, and psresize, for adjusting page sizes.")
>                       (substitute* "base/gscdef.c"
>                         (("GS_DOCDIR")
>                          "\"~/.guix-profile/share/doc/ghostscript\""))))
> +        (add-after 'configure 'remove-timestamps
> +                   (lambda _
> +                     (substitute* "base/mkromfs.c"
> +                       (("time\\(NULL\\)") "0U"))
> +                     #t))
>          (replace 'build
>            (lambda _
>              ;; Build 'libgs.so', but don't build the statically-linked 'gs'
> 

is it possible to patch this in a snippet?

-- 
Efraim Flashner   <efraim <at> flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[signature.asc (application/pgp-signature, inline)]

Message #14 received at 27563 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: 27563 <at> debbugs.gnu.org
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>
Subject: [PATCH v2] gnu: ghostscript: Remove timestamps.
Date: Mon,  3 Jul 2017 21:46:57 +0200

* gnu/packages/ghostscript.scm (ghostscript)[source]: Remove timestamps.
---
 gnu/packages/ghostscript.scm | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/gnu/packages/ghostscript.scm b/gnu/packages/ghostscript.scm
index 1cb651c96..23a86a65a 100644
--- a/gnu/packages/ghostscript.scm
+++ b/gnu/packages/ghostscript.scm
@@ -149,10 +149,14 @@ printing, and psresize, for adjusting page sizes.")
                                      "ghostscript-runpath.patch"))
             (modules '((guix build utils)))
             (snippet
-             ;; Honor --docdir.
-             '(substitute* "Makefile.in"
+             '(begin
+               ;; Get rid of timestamps.
+               (substitute* "base/mkromfs.c"
+                (("time\\(NULL\\)") "0U"))
+               ;; Honor --docdir.
+               (substitute* "Makefile.in"
                 (("^docdir=.*$") "docdir = @docdir@\n")
-                (("^exdir=.*$") "exdir = $(docdir)/examples\n")))))
+                (("^exdir=.*$") "exdir = $(docdir)/examples\n"))))))
    (build-system gnu-build-system)
    (outputs '("out" "doc"))                  ;16 MiB of HTML/PS doc + examples
    (inputs `(("freetype" ,freetype)

Message #17 received at 27563 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 27563 <at> debbugs.gnu.org
Subject: Re: bug#27563: [PATCH] gnu: ghostscript: Add phase
 "remove-timestamps".
Date: Tue, 4 Jul 2017 15:08:34 -0400

[Message part 1 (text/plain, inline)]

On Mon, Jul 03, 2017 at 08:16:46PM +0200, Danny Milosavljevic wrote:
> * gnu/packages/ghostscript.scm (ghostscript): Add phase "remove-timestamps".

Same feedback as for netpbm: If ghostscript works with this change and
we haven't built it yet for core-updates on Hydra, it's okay to make
this change.

But I'm not reviewing the details of the change itself; I'm technically
on vacation this week ;)

[signature.asc (application/pgp-signature, inline)]

Message #20 received at 27563 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Efraim Flashner <efraim <at> flashner.co.il>
Cc: 27563 <at> debbugs.gnu.org, Danny Milosavljevic <dannym <at> scratchpost.org>
Subject: Re: bug#27563: [PATCH] gnu: ghostscript: Add phase
 "remove-timestamps".
Date: Wed, 05 Jul 2017 23:57:06 +0200

Efraim Flashner <efraim <at> flashner.co.il> skribis:

> On Mon, Jul 03, 2017 at 08:16:46PM +0200, Danny Milosavljevic wrote:
>> * gnu/packages/ghostscript.scm (ghostscript): Add phase "remove-timestamps".
>> ---
>>  gnu/packages/ghostscript.scm | 5 +++++
>>  1 file changed, 5 insertions(+)
>> 
>> diff --git a/gnu/packages/ghostscript.scm b/gnu/packages/ghostscript.scm
>> index 1cb651c96..56e05cd77 100644
>> --- a/gnu/packages/ghostscript.scm
>> +++ b/gnu/packages/ghostscript.scm
>> @@ -185,6 +185,11 @@ printing, and psresize, for adjusting page sizes.")
>>                       (substitute* "base/gscdef.c"
>>                         (("GS_DOCDIR")
>>                          "\"~/.guix-profile/share/doc/ghostscript\""))))
>> +        (add-after 'configure 'remove-timestamps
>> +                   (lambda _
>> +                     (substitute* "base/mkromfs.c"
>> +                       (("time\\(NULL\\)") "0U"))
>> +                     #t))
>>          (replace 'build
>>            (lambda _
>>              ;; Build 'libgs.so', but don't build the statically-linked 'gs'
>> 
>
> is it possible to patch this in a snippet?

Agreed; your second version LGTM.

Like Leo, I’d invite you to check the state of ‘core-updates’.  :-)
Depending on how far it went, perhaps we can push it there along with
the netpbm one, cancel pending builds, and start a new evaluation (let
me know if you want to do these.)

Thanks,
Ludo’.

Message #27 received at 27563 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: 27563 <at> debbugs.gnu.org
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>
Subject: [PATCH] gnu: ghostscript: Don't write document UUID;
 use "" as instance UUID.
Date: Thu,  6 Jul 2017 11:07:54 +0200

* gnu/packages/patches/ghostscript-no-header-uuid.patch: New file.
* gnu/local.mk (dist_patch_DATA): Add it.
* gnu/packages/ghostscript.scm (ghostscript)[source]: Use it.
---
 gnu/local.mk                                       |  1 +
 gnu/packages/ghostscript.scm                       |  3 ++-
 .../patches/ghostscript-no-header-uuid.patch       | 28 ++++++++++++++++++++++
 3 files changed, 31 insertions(+), 1 deletion(-)
 create mode 100644 gnu/packages/patches/ghostscript-no-header-uuid.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index 900df572a..9eb6ac2d6 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -620,6 +620,7 @@ dist_patch_DATA =						\
   %D%/packages/patches/ghc-dont-pass-linker-flags-via-response-files.patch	\
   %D%/packages/patches/ghostscript-CVE-2017-8291.patch		\
   %D%/packages/patches/ghostscript-runpath.patch		\
+  %D%/packages/patches/ghostscript-no-header-uuid.patch		\
   %D%/packages/patches/glib-networking-ssl-cert-file.patch	\
   %D%/packages/patches/glib-tests-timer.patch			\
   %D%/packages/patches/glibc-CVE-2017-1000366-pt1.patch		\
diff --git a/gnu/packages/ghostscript.scm b/gnu/packages/ghostscript.scm
index b9ba1c081..0b2787c68 100644
--- a/gnu/packages/ghostscript.scm
+++ b/gnu/packages/ghostscript.scm
@@ -143,7 +143,8 @@ printing, and psresize, for adjusting page sizes.")
          (base32
           "0lyhjcrkmd5fcmh8h56bs4xr9k4jasmikv5vsix1hd4ai0ad1q9b"))
         (patches (search-patches "ghostscript-runpath.patch"
-                                 "ghostscript-CVE-2017-8291.patch"))
+                                 "ghostscript-CVE-2017-8291.patch"
+                                 "ghostscript-no-header-uuid.patch"))
         (modules '((guix build utils)))
         (snippet
           ;; Remove bundled libraries. The bundled OpenJPEG is a patched fork so
diff --git a/gnu/packages/patches/ghostscript-no-header-uuid.patch b/gnu/packages/patches/ghostscript-no-header-uuid.patch
new file mode 100644
index 000000000..5f13e285b
--- /dev/null
+++ b/gnu/packages/patches/ghostscript-no-header-uuid.patch
@@ -0,0 +1,28 @@
+--- a/devices/vector/gdevpdfe.c	2017-07-06 10:22:28.131889858 +0200
++++ b/devices/vector/gdevpdfe.c	2017-07-06 10:24:25.137842222 +0200
+@@ -612,7 +612,7 @@
+         return code;
+ 
+     /* PDF/A XMP reference recommends setting UUID to empty. If not empty must be a URI */
+-    if (pdev->PDFA != 0)
++    if (pdev->PDFA != 0 || 1)
+         instance_uuid[0] = 0x00;
+ 
+     cre_date_time_len = pdf_get_docinfo_item(pdev, "/CreationDate", cre_date_time, sizeof(cre_date_time));
+@@ -715,16 +715,6 @@
+             pdf_xml_newline(s);
+ 
+             pdf_xml_tag_open_beg(s, "rdf:Description");
+-            pdf_xml_attribute_name(s, "rdf:about");
+-            pdf_xml_attribute_value(s, instance_uuid);
+-            pdf_xml_attribute_name(s, "xmlns:xapMM");
+-            pdf_xml_attribute_value(s, "http://ns.adobe.com/xap/1.0/mm/");
+-            pdf_xml_attribute_name(s, "xapMM:DocumentID");
+-            pdf_xml_attribute_value(s, document_uuid);
+-            pdf_xml_tag_end_empty(s);
+-            pdf_xml_newline(s);
+-
+-            pdf_xml_tag_open_beg(s, "rdf:Description");
+             pdf_xml_attribute_name(s, "rdf:about");
+             pdf_xml_attribute_value(s, instance_uuid);
+             pdf_xml_attribute_name(s, "xmlns:dc");

Message #30 received at 27563 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: 27563 <at> debbugs.gnu.org
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>
Subject: [PATCH v3 1/2] gnu: ghostscript: Don't write document UUID;
 use "" as instance UUID.
Date: Thu,  6 Jul 2017 12:32:15 +0200

* gnu/packages/patches/ghostscript-no-header-uuid.patch: New file.
* gnu/local.mk (dist_patch_DATA): Add it.
* gnu/packages/ghostscript.scm (ghostscript)[source]: Use it.
---
 gnu/local.mk                                       |  1 +
 gnu/packages/ghostscript.scm                       |  3 ++-
 .../patches/ghostscript-no-header-uuid.patch       | 28 ++++++++++++++++++++++
 3 files changed, 31 insertions(+), 1 deletion(-)
 create mode 100644 gnu/packages/patches/ghostscript-no-header-uuid.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index 900df572a..9eb6ac2d6 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -620,6 +620,7 @@ dist_patch_DATA =						\
   %D%/packages/patches/ghc-dont-pass-linker-flags-via-response-files.patch	\
   %D%/packages/patches/ghostscript-CVE-2017-8291.patch		\
   %D%/packages/patches/ghostscript-runpath.patch		\
+  %D%/packages/patches/ghostscript-no-header-uuid.patch		\
   %D%/packages/patches/glib-networking-ssl-cert-file.patch	\
   %D%/packages/patches/glib-tests-timer.patch			\
   %D%/packages/patches/glibc-CVE-2017-1000366-pt1.patch		\
diff --git a/gnu/packages/ghostscript.scm b/gnu/packages/ghostscript.scm
index b9ba1c081..0b2787c68 100644
--- a/gnu/packages/ghostscript.scm
+++ b/gnu/packages/ghostscript.scm
@@ -143,7 +143,8 @@ printing, and psresize, for adjusting page sizes.")
          (base32
           "0lyhjcrkmd5fcmh8h56bs4xr9k4jasmikv5vsix1hd4ai0ad1q9b"))
         (patches (search-patches "ghostscript-runpath.patch"
-                                 "ghostscript-CVE-2017-8291.patch"))
+                                 "ghostscript-CVE-2017-8291.patch"
+                                 "ghostscript-no-header-uuid.patch"))
         (modules '((guix build utils)))
         (snippet
           ;; Remove bundled libraries. The bundled OpenJPEG is a patched fork so
diff --git a/gnu/packages/patches/ghostscript-no-header-uuid.patch b/gnu/packages/patches/ghostscript-no-header-uuid.patch
new file mode 100644
index 000000000..5f13e285b
--- /dev/null
+++ b/gnu/packages/patches/ghostscript-no-header-uuid.patch
@@ -0,0 +1,28 @@
+--- a/devices/vector/gdevpdfe.c	2017-07-06 10:22:28.131889858 +0200
++++ b/devices/vector/gdevpdfe.c	2017-07-06 10:24:25.137842222 +0200
+@@ -612,7 +612,7 @@
+         return code;
+ 
+     /* PDF/A XMP reference recommends setting UUID to empty. If not empty must be a URI */
+-    if (pdev->PDFA != 0)
++    if (pdev->PDFA != 0 || 1)
+         instance_uuid[0] = 0x00;
+ 
+     cre_date_time_len = pdf_get_docinfo_item(pdev, "/CreationDate", cre_date_time, sizeof(cre_date_time));
+@@ -715,16 +715,6 @@
+             pdf_xml_newline(s);
+ 
+             pdf_xml_tag_open_beg(s, "rdf:Description");
+-            pdf_xml_attribute_name(s, "rdf:about");
+-            pdf_xml_attribute_value(s, instance_uuid);
+-            pdf_xml_attribute_name(s, "xmlns:xapMM");
+-            pdf_xml_attribute_value(s, "http://ns.adobe.com/xap/1.0/mm/");
+-            pdf_xml_attribute_name(s, "xapMM:DocumentID");
+-            pdf_xml_attribute_value(s, document_uuid);
+-            pdf_xml_tag_end_empty(s);
+-            pdf_xml_newline(s);
+-
+-            pdf_xml_tag_open_beg(s, "rdf:Description");
+             pdf_xml_attribute_name(s, "rdf:about");
+             pdf_xml_attribute_value(s, instance_uuid);
+             pdf_xml_attribute_name(s, "xmlns:dc");

Message #33 received at 27563 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: 27563 <at> debbugs.gnu.org
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>
Subject: [PATCH v3 0/2] Make ghostscript reproducible.
Date: Thu,  6 Jul 2017 12:32:14 +0200

So this is what's needed to finally make ghostscript, netpbm and groff
reproducible.  Groff just finished its 38th build on my machine and it
finally compared the rounds as equal.

I'm posting those here in order to make sure we all agree that this is
the way to go.

The patchset patches PDF creation in ghostscript.  It's for core-updates.

The PDF file has a trailer field "/ID" which is required only when
encrypting.  But ghostscript derives it from the current time.
So I figured leaving it off if allowed would be the easiest fix.
If it's not there then it can't change :P

Also, newer PDF files have an RDF header specifying some extra information
in an XML-like format.  For example there's an instance UUID (PDF/A specifies
that it's recommended to set this to an empty string), and a document UUID.
The latter again is time-based.

This patchset
* removes the RDF tag which contains the document UUID and
* sets the instance UUID to "" and
* removes the ID tag if allowed (i.e. if not encrypting).

Because of the printf-style functions, it has to split up the printfs a bit,
but really it just makes one of the parts printed optional - in multiple
places (because PDF trailers can be chained).

Danny Milosavljevic (2):
  gnu: ghostscript: Don't write document UUID; use "" as instance UUID.
  gnu: ghostscript: Write document ID only when encrypting.

 gnu/local.mk                                       |  2 +
 gnu/packages/ghostscript.scm                       |  4 +-
 .../patches/ghostscript-no-header-id.patch         | 47 ++++++++++++++++++++++
 .../patches/ghostscript-no-header-uuid.patch       | 28 +++++++++++++
 4 files changed, 80 insertions(+), 1 deletion(-)
 create mode 100644 gnu/packages/patches/ghostscript-no-header-id.patch
 create mode 100644 gnu/packages/patches/ghostscript-no-header-uuid.patch

Message #36 received at 27563 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: 27563 <at> debbugs.gnu.org
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>
Subject: [PATCH v3 2/2] gnu: ghostscript: Write document ID only when
 encrypting.
Date: Thu,  6 Jul 2017 12:32:16 +0200

* gnu/packages/patches/ghostscript-no-header-id.patch: New file.
* gnu/local.mk (dist_patch_DATA): Add it.
* gnu/packages/ghostscript.scm (ghostscript)[source]: Use it.
---
 gnu/local.mk                                       |  1 +
 gnu/packages/ghostscript.scm                       |  1 +
 .../patches/ghostscript-no-header-id.patch         | 47 ++++++++++++++++++++++
 3 files changed, 49 insertions(+)
 create mode 100644 gnu/packages/patches/ghostscript-no-header-id.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index 9eb6ac2d6..935794550 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -620,6 +620,7 @@ dist_patch_DATA =						\
   %D%/packages/patches/ghc-dont-pass-linker-flags-via-response-files.patch	\
   %D%/packages/patches/ghostscript-CVE-2017-8291.patch		\
   %D%/packages/patches/ghostscript-runpath.patch		\
+  %D%/packages/patches/ghostscript-no-header-id.patch		\
   %D%/packages/patches/ghostscript-no-header-uuid.patch		\
   %D%/packages/patches/glib-networking-ssl-cert-file.patch	\
   %D%/packages/patches/glib-tests-timer.patch			\
diff --git a/gnu/packages/ghostscript.scm b/gnu/packages/ghostscript.scm
index 0b2787c68..f983fb20b 100644
--- a/gnu/packages/ghostscript.scm
+++ b/gnu/packages/ghostscript.scm
@@ -144,6 +144,7 @@ printing, and psresize, for adjusting page sizes.")
           "0lyhjcrkmd5fcmh8h56bs4xr9k4jasmikv5vsix1hd4ai0ad1q9b"))
         (patches (search-patches "ghostscript-runpath.patch"
                                  "ghostscript-CVE-2017-8291.patch"
+                                 "ghostscript-no-header-id.patch"
                                  "ghostscript-no-header-uuid.patch"))
         (modules '((guix build utils)))
         (snippet
diff --git a/gnu/packages/patches/ghostscript-no-header-id.patch b/gnu/packages/patches/ghostscript-no-header-id.patch
new file mode 100644
index 000000000..3526a300e
--- /dev/null
+++ b/gnu/packages/patches/ghostscript-no-header-id.patch
@@ -0,0 +1,47 @@
+--- ./devices/vector/gdevpdf.c.orig	2017-07-06 11:08:33.671196250 +0200
++++ ./devices/vector/gdevpdf.c	2017-07-06 11:39:25.282006143 +0200
+@@ -1759,8 +1759,11 @@
+      * +1 for the linearisation dict and +1 for the primary hint stream.
+      */
+     linear_params->FirsttrailerOffset = gp_ftell_64(linear_params->Lin_File.file);
+-    gs_sprintf(LDict, "\ntrailer\n<</Size %ld/Info %d 0 R/Root %d 0 R/ID[%s%s]/Prev %d>>\nstartxref\r\n0\n%%%%EOF\n        \n",
+-        linear_params->LastResource + 3, pdev->ResourceUsage[linear_params->Info_id].NewObjectNumber, pdev->ResourceUsage[linear_params->Catalog_id].NewObjectNumber, fileID, fileID, 0);
++    gs_sprintf(LDict, "\ntrailer\n<</Size %ld/Info %d 0 R/Root %d 0 R",
++        linear_params->LastResource + 3, pdev->ResourceUsage[linear_params->Info_id].NewObjectNumber, pdev->ResourceUsage[linear_params->Catalog_id].NewObjectNumber);
++    if (pdev->OwnerPassword.size > 0) /* ID is mandatory when encrypting */
++        gs_sprintf(LDict, "/ID[%s%s]", fileID, fileID);
++    gs_sprintf(LDict, "/Prev %d>>\nstartxref\r\n0\n%%%%EOF\n        \n", 0);
+     fwrite(LDict, strlen(LDict), 1, linear_params->Lin_File.file);
+ 
+     /* Write document catalog (Part 4) */
+@@ -2292,8 +2295,11 @@
+      * in the missing values.
+      */
+     code = gp_fseek_64(linear_params->sfile, linear_params->FirsttrailerOffset, SEEK_SET);
+-    gs_sprintf(LDict, "\ntrailer\n<</Size %ld/Info %d 0 R/Root %d 0 R/ID[%s%s]/Prev %"PRId64">>\nstartxref\r\n0\n%%%%EOF\n",
+-        linear_params->LastResource + 3, pdev->ResourceUsage[linear_params->Info_id].NewObjectNumber, pdev->ResourceUsage[linear_params->Catalog_id].NewObjectNumber, fileID, fileID, mainxref);
++    gs_sprintf(LDict, "\ntrailer\n<</Size %ld/Info %d 0 R/Root %d 0 R",
++        linear_params->LastResource + 3, pdev->ResourceUsage[linear_params->Info_id].NewObjectNumber, pdev->ResourceUsage[linear_params->Catalog_id].NewObjectNumber);
++    if (pdev->OwnerPassword.size > 0) /* ID is mandatory when encrypting */
++        gs_sprintf(LDict, "/ID[%s%s]", fileID, fileID);
++    gs_sprintf(LDict, "/Prev %"PRId64">>\nstartxref\r\n0\n%%%%EOF\n", mainxref);
+     fwrite(LDict, strlen(LDict), 1, linear_params->sfile);
+ 
+     code = gp_fseek_64(linear_params->sfile, pdev->ResourceUsage[HintStreamObj].LinearisedOffset, SEEK_SET);
+@@ -2930,10 +2936,12 @@
+             stream_puts(s, "trailer\n");
+             pprintld3(s, "<< /Size %ld /Root %ld 0 R /Info %ld 0 R\n",
+                   pdev->next_id, Catalog_id, Info_id);
+-            stream_puts(s, "/ID [");
+-            psdf_write_string(pdev->strm, pdev->fileID, sizeof(pdev->fileID), 0);
+-            psdf_write_string(pdev->strm, pdev->fileID, sizeof(pdev->fileID), 0);
+-            stream_puts(s, "]\n");
++            if (pdev->OwnerPassword.size > 0) { /* ID is mandatory when encrypting */
++                stream_puts(s, "/ID [");
++                psdf_write_string(pdev->strm, pdev->fileID, sizeof(pdev->fileID), 0);
++                psdf_write_string(pdev->strm, pdev->fileID, sizeof(pdev->fileID), 0);
++                stream_puts(s, "]\n");
++            }
+             if (pdev->OwnerPassword.size > 0) {
+                 pprintld1(s, "/Encrypt %ld 0 R ", Encrypt_id);
+             }

Message #39 received at 27563 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: 27563 <at> debbugs.gnu.org
Subject: Re: bug#27563: ghostscript-with-cups is not reproducible
Date: Thu, 6 Jul 2017 12:59:17 +0200

Upstream bug report: <https://bugs.ghostscript.com/show_bug.cgi?id=698208>.

Message #42 received at 27563 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 27563 <at> debbugs.gnu.org
Subject: Re: bug#27563: [PATCH] gnu: ghostscript: Don't write document UUID;
 use "" as instance UUID.
Date: Fri, 07 Jul 2017 13:55:16 +0200

Danny Milosavljevic <dannym <at> scratchpost.org> skribis:

> * gnu/packages/patches/ghostscript-no-header-uuid.patch: New file.
> * gnu/local.mk (dist_patch_DATA): Add it.
> * gnu/packages/ghostscript.scm (ghostscript)[source]: Use it.

[...]

> --- /dev/null
> +++ b/gnu/packages/patches/ghostscript-no-header-uuid.patch
> @@ -0,0 +1,28 @@
> +--- a/devices/vector/gdevpdfe.c	2017-07-06 10:22:28.131889858 +0200
> ++++ b/devices/vector/gdevpdfe.c	2017-07-06 10:24:25.137842222 +0200

Please add a comment at the top of the patch to explain what it does and
what its upstream status is or if there were discussions around it.

> +             pdf_xml_tag_open_beg(s, "rdf:Description");
> +-            pdf_xml_attribute_name(s, "rdf:about");
> +-            pdf_xml_attribute_value(s, instance_uuid);
> +-            pdf_xml_attribute_name(s, "xmlns:xapMM");
> +-            pdf_xml_attribute_value(s, "http://ns.adobe.com/xap/1.0/mm/");
> +-            pdf_xml_attribute_name(s, "xapMM:DocumentID");
> +-            pdf_xml_attribute_value(s, document_uuid);
> +-            pdf_xml_tag_end_empty(s);
> +-            pdf_xml_newline(s);
> +-
> +-            pdf_xml_tag_open_beg(s, "rdf:Description");

What’s the impact of simply removing all of this from generated PDF
files?  Should we instead make it optional?  For instance, we could
introduce a new environment variable, say “GHOSTSCRIPT_PDF_UUID”, and
use that as the document UUID when it’s defined.  In our build
environments, we’d always set GHOSTSCRIPT_PDF_UUID to a known value to
ensure determinism.

WDYT?

Also, what does Debian do?  :-)

Thank you for working on it!

Ludo’.

Message #45 received at 27563 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 27563 <at> debbugs.gnu.org
Subject: Re: bug#27563: [PATCH v3 0/2] Make ghostscript reproducible.
Date: Fri, 07 Jul 2017 14:00:09 +0200

Danny Milosavljevic <dannym <at> scratchpost.org> skribis:

> Also, newer PDF files have an RDF header specifying some extra information
> in an XML-like format.  For example there's an instance UUID (PDF/A specifies
> that it's recommended to set this to an empty string), and a document UUID.
> The latter again is time-based.

If it’s time-based, then the solution may be to honor SOURCE_DATE_EPOCH.

I asked on #reproducible-builds (OFTC).  A patch had been proposed
upstream but rejected:

  http://bugs.ghostscript.com/show_bug.cgi?id=696765

See also
<https://wiki.debian.org/ReproducibleBuilds/PdfGeneratedByGhostscript>.

Ludo’.

Message #48 received at 27563 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 27563 <at> debbugs.gnu.org
Subject: Re: bug#27563: [PATCH v3 2/2] gnu: ghostscript: Write document ID
 only when encrypting.
Date: Fri, 07 Jul 2017 14:02:04 +0200

Danny Milosavljevic <dannym <at> scratchpost.org> skribis:

> * gnu/packages/patches/ghostscript-no-header-id.patch: New file.
> * gnu/local.mk (dist_patch_DATA): Add it.
> * gnu/packages/ghostscript.scm (ghostscript)[source]: Use it.

[...]

> new file mode 100644
> index 000000000..3526a300e
> --- /dev/null
> +++ b/gnu/packages/patches/ghostscript-no-header-id.patch

Please add an explanation here.

Also, do you know whether the PDF specs are OK with that?  Might be good
to discuss with upstream, we wouldn’t want to generate somewhat broken
PDFs.  WDYT?

Thank you,
Ludo’.

Message #51 received at 27563 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 27563 <at> debbugs.gnu.org
Subject: Re: bug#27563: [PATCH v3 2/2] gnu: ghostscript: Write document ID
 only when encrypting.
Date: Fri, 7 Jul 2017 15:21:49 +0200

Hi Ludo,

On Fri, 07 Jul 2017 14:02:04 +0200
ludo <at> gnu.org (Ludovic Courtès) wrote:

> Also, do you know whether the PDF specs are OK with that?  

Yeah, at the upstream bug link <https://bugs.ghostscript.com/show_bug.cgi?id=698208> we discussed that (somewhat).  While they don't want to carry the patches (because they don't want to lose functionality) they explained that it might well be that *future* versions of the spec could make ID and UUID mandatory.

Right now there's a stringent spec, called PDF/A (for "archiving"; which is intended for governing bodies where you don't want existing documents that dynamically alter their contents after some time - like with Javascript or something) which already sets the instance UUID to "".  So I just set it to "" always rather than just for PDF/A.

Also, as far as I understand the "/ID" is currently only mandatory when encrypting, although in the future it might change.

That leaves the document UUID - and upstream, in some of the other bugreports, explained that they want UNIQUE document UUIDs.  So I figured that we should just leave it off - so it's not the same over multiple documents.  They are definitely not fine with non-unique UUIDs.

This RDF metadata stuff (the instance UUID and document UUID) is quite new.  In a former life I wrote PDF parsers and I didn't handle the RDF back then at all.  So I guess it would even work to leave the entire RDF metadata off - after all, it worked back then.

If someone is well-versed in XMP RDF metadata for PDF, I wonder what is better: leaving the entire RDF off or just leaving the element containing the document id (as an attribute) off.  Currently, the patch does the latter.  The specification by adobe (XMP Specification Part 1, ISO 16684-1:2011(E) Annex A) says "The use of robust GUIDs is encouraged; having globally unique values is important" but as far as I can see doesn't say whether they are mandatory.

I also thought of patching groff instead.  But it seems that groff is now searching for a maintainer - I'm not sure anyone would integrate it there.  Also, I'm not well-versed in perl.  Also, patching finished PDFs (using regexps or something) is kinda dangerous because nobody *forces* you to encode the streams (think: attachements) in PDFs.  So it could be that some other non-PDF thing is integrated into the PDF as a stream and the regexp substituter would just substitute it in there as well.

There's a program "pdfmark" which is supposed to be for changing the metadata for PDFs but upstream said that it can't change those fields.  It could change the CreationDate, ModDate etc.

In short, I think the lowest risk is patching ghostscript as we did here.

Message #54 received at 27563 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 27563 <at> debbugs.gnu.org
Subject: Re: bug#27563: [PATCH v3 0/2] Make ghostscript reproducible.
Date: Fri, 7 Jul 2017 15:25:38 +0200

Hi Ludo,

On Fri, 07 Jul 2017 14:00:09 +0200
ludo <at> gnu.org (Ludovic Courtès) wrote:

> Danny Milosavljevic <dannym <at> scratchpost.org> skribis:
> 
> > Also, newer PDF files have an RDF header specifying some extra information
> > in an XML-like format.  For example there's an instance UUID (PDF/A specifies
> > that it's recommended to set this to an empty string), and a document UUID.
> > The latter again is time-based.  
> 
> If it’s time-based, then the solution may be to honor SOURCE_DATE_EPOCH.

Upstream says definitely not.  The UUIDs are supposed to be unique and they don't want anyone writing fixed UUIDs into documents (except for "" for the instance ID which they themselves do).

I think there could be some enterprise search engine which associates a document with other resources using the document UUID - and if everyone went and reused UUIDs it would be very confused.

That's why I left it off.

Message #57 received at 27563 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 27563 <at> debbugs.gnu.org
Subject: Re: bug#27563: [PATCH] gnu: ghostscript: Don't write document UUID;
 use "" as instance UUID.
Date: Fri, 7 Jul 2017 15:28:04 +0200

> What’s the impact of simply removing all of this from generated PDF
> files?  Should we instead make it optional?

I don't know.  It's just one metadata element - and it's recent.  I mean that there are lots of (old) PDF files that don't have it in the first place.

>  For instance, we could
> introduce a new environment variable, say “GHOSTSCRIPT_PDF_UUID”, and
> use that as the document UUID when it’s defined.  

>>In our build
> environments, we’d always set GHOSTSCRIPT_PDF_UUID to a known value to
> ensure determinism.

Upstream definitely doesn't want that - at least not if it's a constant value for all PDFs.

Message #60 received at 27563 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 27563 <at> debbugs.gnu.org
Subject: Re: bug#27563: [PATCH v3 0/2] Make ghostscript reproducible.
Date: Fri, 07 Jul 2017 17:18:15 +0200

Danny Milosavljevic <dannym <at> scratchpost.org> skribis:

> Hi Ludo,
>
> On Fri, 07 Jul 2017 14:00:09 +0200
> ludo <at> gnu.org (Ludovic Courtès) wrote:
>
>> Danny Milosavljevic <dannym <at> scratchpost.org> skribis:
>> 
>> > Also, newer PDF files have an RDF header specifying some extra information
>> > in an XML-like format.  For example there's an instance UUID (PDF/A specifies
>> > that it's recommended to set this to an empty string), and a document UUID.
>> > The latter again is time-based.  
>> 
>> If it’s time-based, then the solution may be to honor SOURCE_DATE_EPOCH.
>
> Upstream says definitely not.  The UUIDs are supposed to be unique and they don't want anyone writing fixed UUIDs into documents (except for "" for the instance ID which they themselves do).
>
> I think there could be some enterprise search engine which associates a document with other resources using the document UUID - and if everyone went and reused UUIDs it would be very confused.
>
> That's why I left it off.

OK, makes sense.  Maybe we can still have it disabled (or enabled) by
environment variable instead of having it removed wholesale?

Ludo’.

Message #63 received at 27563 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 27563 <at> debbugs.gnu.org
Subject: Re: bug#27563: [PATCH v3 0/2] Make ghostscript reproducible.
Date: Fri, 7 Jul 2017 17:34:17 +0200

> <https://wiki.debian.org/ReproducibleBuilds/PdfGeneratedByGhostscript>.

Hmm... can you access the patch linked there (under "Solution") ?

Message #66 received at 27563 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 27563 <at> debbugs.gnu.org
Subject: Re: bug#27563: [PATCH v3 0/2] Make ghostscript reproducible.
Date: Fri, 7 Jul 2017 17:44:22 +0200

On Fri, 07 Jul 2017 17:18:15 +0200
ludo <at> gnu.org (Ludovic Courtès) wrote:

> OK, makes sense.  Maybe we can still have it disabled (or enabled) by
> environment variable

Sure.  Any suggestions for the name of the environment variable?  Also, where would we set it so the build processes of all the other packages actually pick it up?

Would it disable and re-enable all these things at once? :

* CreationDate
* ModDate
* /ID
* XMP DocumentUUID
* XMP InstanceUUID

Message #69 received at 27563 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 27563 <at> debbugs.gnu.org, Ludovic Courtès <ludo <at> gnu.org>
Subject: Re: bug#27563: [PATCH v3 2/2] gnu: ghostscript: Write document ID
 only when encrypting.
Date: Fri, 7 Jul 2017 12:21:51 -0400

[Message part 1 (text/plain, inline)]

On Fri, Jul 07, 2017 at 03:21:49PM +0200, Danny Milosavljevic wrote:
> Yeah, at the upstream bug link
> <https://bugs.ghostscript.com/show_bug.cgi?id=698208> we discussed
> that (somewhat).  While they don't want to carry the patches (because
> they don't want to lose functionality) they explained that it might
> well be that *future* versions of the spec could make ID and UUID
> mandatory.
> 
> Right now there's a stringent spec, called PDF/A (for "archiving";
> which is intended for governing bodies where you don't want existing
> documents that dynamically alter their contents after some time - like
> with Javascript or something) which already sets the instance UUID to
> "".  So I just set it to "" always rather than just for PDF/A.
> 
> Also, as far as I understand the "/ID" is currently only mandatory
> when encrypting, although in the future it might change.
> 
> That leaves the document UUID - and upstream, in some of the other
> bugreports, explained that they want UNIQUE document UUIDs.  So I
> figured that we should just leave it off - so it's not the same over
> multiple documents.  They are definitely not fine with non-unique
> UUIDs.
> 
> This RDF metadata stuff (the instance UUID and document UUID) is quite
> new.  In a former life I wrote PDF parsers and I didn't handle the RDF
> back then at all.  So I guess it would even work to leave the entire
> RDF metadata off - after all, it worked back then.
> 
> If someone is well-versed in XMP RDF metadata for PDF, I wonder what
> is better: leaving the entire RDF off or just leaving the element
> containing the document id (as an attribute) off.  Currently, the
> patch does the latter.  The specification by adobe (XMP Specification
> Part 1, ISO 16684-1:2011(E) Annex A) says "The use of robust GUIDs is
> encouraged; having globally unique values is important" but as far as
> I can see doesn't say whether they are mandatory.
> 
> I also thought of patching groff instead.  But it seems that groff is
> now searching for a maintainer - I'm not sure anyone would integrate
> it there.  Also, I'm not well-versed in perl.  Also, patching finished
> PDFs (using regexps or something) is kinda dangerous because nobody
> *forces* you to encode the streams (think: attachements) in PDFs.  So
> it could be that some other non-PDF thing is integrated into the PDF
> as a stream and the regexp substituter would just substitute it in
> there as well.
> 
> There's a program "pdfmark" which is supposed to be for changing the
> metadata for PDFs but upstream said that it can't change those fields.
> It could change the CreationDate, ModDate etc.
> 
> In short, I think the lowest risk is patching ghostscript as we did
> here.

I think the lowest risk is to do nothing to Ghostscript and move the PDF
documentation to a separate 'doc' output. Then, we could have
reproducible binaries and ignore the PDF issues for now. Does anyone
know how many packages include PDF documentation built with Ghostscript?

I think the next lowest risk is to do nothing.

I think it's risky to patch Ghostscript, for a few reasons:

1) The patches don't include provenance information, so it's difficult
to find any other discussion of them. I'd like for the Ghostscript
maintainers to have reviewed the proposed changes, both for code
correctness and for PDF-specific issues.
2) At least some of the patches in the related Ghostscript discussions
seem to be proof of concepts rather than finished code:
https://bugs.ghostscript.com/show_bug.cgi?id=697484#c3
So, if these patches came from there, we'd want to be extra careful.

By the way, this is the patch used for Debian's latest Ghostscript
package:

https://anonscm.debian.org/git/printing/ghostscript.git/tree/debian/patches/2010_add_build_timestamp_setting.patch?id=e2bf3ad7026afe13636d4937430c3fdae7854078

That patch was not reviewed on a public forum, at least nothing I can
find with Google. Again, I'd want to get the Ghostscript team's advice.

[signature.asc (application/pgp-signature, inline)]

Message #72 received at 27563 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: Leo Famulari <leo <at> famulari.name>
Cc: 27563 <at> debbugs.gnu.org, Ludovic Courtès <ludo <at> gnu.org>
Subject: Re: bug#27563: [PATCH v3 2/2] gnu: ghostscript: Write document ID
 only when encrypting.
Date: Fri, 7 Jul 2017 18:42:25 +0200

Hi Leo,

On Fri, 7 Jul 2017 12:21:51 -0400
Leo Famulari <leo <at> famulari.name> wrote:

> > That leaves the document UUID - and upstream, in some of the other
> I think the lowest risk is to do nothing to Ghostscript and move the PDF
> documentation to a separate 'doc' output. Then, we could have
> reproducible binaries and ignore the PDF issues for now. Does anyone
> know how many packages include PDF documentation built with Ghostscript?

Aren't the derivations of the doc outputs still a problem?  For example, Hydra will run out of space sooner or later because it keeps building them, right?

> 2) At least some of the patches in the related Ghostscript discussions
> seem to be proof of concepts rather than finished code:
> https://bugs.ghostscript.com/show_bug.cgi?id=697484#c3
> So, if these patches came from there, we'd want to be extra careful.

No, I wrote the ones here without external sources (except for the direct discussion on my newish upstream bug report, and the PDF and XMP specifications - whatever worth they have).

> By the way, this is the patch used for Debian's latest Ghostscript
> package:
> 
> https://anonscm.debian.org/git/printing/ghostscript.git/tree/debian/patches/2010_add_build_timestamp_setting.patch?id=e2bf3ad7026afe13636d4937430c3fdae7854078
> 
> That patch was not reviewed on a public forum, at least nothing I can
> find with Google. Again, I'd want to get the Ghostscript team's advice.

On such an approach they advised that we should only generate *unique* UUIDs.  But the UUIDs are generated from these times.  So that linked patch would generate multiple non-unique uuids on systems.

That's why I removed the entire UUID and Time sections and actually didn't fiddle with the ghostscript-internal times at all.  Builds reproducibly.

I wonder how many packages actually use the ghostscript pdf writer too.  How to find that out?

Note that groff itself also fails to build reproducibly without the patches.

In any case, the patch 2/2 is quite tame (it looks scary because of the printf splitting, but it's actually just either leaving "/ID[...]" off or not, globally).

But I understand that it would be even easier to do nothing.  Wouldn't make the stuff reproducible, though.

I'd vote for an environment variable to disable UUID printing and also Time header printing.  That way it would do everything normally in regular usage - but when used in packages, it would just not *print* the problematic stuff.  No internal state is changed at all by the patches.

Message #75 received at 27563 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 27563 <at> debbugs.gnu.org, Ludovic Courtès <ludo <at> gnu.org>
Subject: Re: bug#27563: [PATCH v3 2/2] gnu: ghostscript: Write document ID
 only when encrypting.
Date: Fri, 7 Jul 2017 13:24:07 -0400

[Message part 1 (text/plain, inline)]

On Fri, Jul 07, 2017 at 06:42:25PM +0200, Danny Milosavljevic wrote:
> Leo Famulari <leo <at> famulari.name> wrote:
> > > That leaves the document UUID - and upstream, in some of the other
> > I think the lowest risk is to do nothing to Ghostscript and move the PDF
> > documentation to a separate 'doc' output. Then, we could have
> > reproducible binaries and ignore the PDF issues for now. Does anyone
> > know how many packages include PDF documentation built with Ghostscript?
> 
> Aren't the derivations of the doc outputs still a problem?  For
> example, Hydra will run out of space sooner or later because it keeps
> building them, right?

Do these timestamps and UUID affect the derivations? I figured they only
affected the result of running the derivation — that is, the output of
the build process. Those outputs are what we'd like to create
reproducibly, but they don't cause rebuilds if they are not
reproducible.

If a package's dependency graph is identical to before, Guix (and I
assume Hydra) will not rebuild it, even if we humans know that the built
output is unreproducible, such as when timestamps are embedded.

My apologies if I misinterpreted your question.

We run out of space and have to garbage collect periodically anyways.
Regardless, once we own the Hydra machine, I'd like for us to buy a huge
amount of storage and keep built outputs for much longer than we do now.
In practice, it's not really possible to go back in time more than 6
months of Guix, due to missing upstream sources and test suites with
expiration dates.

> > 2) At least some of the patches in the related Ghostscript discussions
> > seem to be proof of concepts rather than finished code:
> > https://bugs.ghostscript.com/show_bug.cgi?id=697484#c3
> > So, if these patches came from there, we'd want to be extra careful.
> 
> No, I wrote the ones here without external sources (except for the
> direct discussion on my newish upstream bug report, and the PDF and
> XMP specifications - whatever worth they have).

Ah, thanks for the clarification.

> > By the way, this is the patch used for Debian's latest Ghostscript
> > package:
> > 
> > https://anonscm.debian.org/git/printing/ghostscript.git/tree/debian/patches/2010_add_build_timestamp_setting.patch?id=e2bf3ad7026afe13636d4937430c3fdae7854078
> > 
> > That patch was not reviewed on a public forum, at least nothing I can
> > find with Google. Again, I'd want to get the Ghostscript team's advice.
> 
> On such an approach they advised that we should only generate *unique*
> UUIDs.  But the UUIDs are generated from these times.  So that linked
> patch would generate multiple non-unique uuids on systems.
> 
> That's why I removed the entire UUID and Time sections and actually
> didn't fiddle with the ghostscript-internal times at all.  Builds
> reproducibly.
> 
> I wonder how many packages actually use the ghostscript pdf writer
> too.  How to find that out?
> 
> Note that groff itself also fails to build reproducibly without the
> patches.
> 
> In any case, the patch 2/2 is quite tame (it looks scary because of
> the printf splitting, but it's actually just either leaving "/ID[...]"
> off or not, globally).
> 
> But I understand that it would be even easier to do nothing.  Wouldn't
> make the stuff reproducible, though.
> 
> I'd vote for an environment variable to disable UUID printing and also
> Time header printing.  That way it would do everything normally in
> regular usage - but when used in packages, it would just not *print*
> the problematic stuff.  No internal state is changed at all by the
> patches.

Okay, thank you for explaining this (especially if you already explained
it! It's hard to join a conversation like this halfway through). I'll
read your patches carefully later today.

[signature.asc (application/pgp-signature, inline)]

Message #78 received at 27563 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 27563 <at> debbugs.gnu.org, Leo Famulari <leo <at> famulari.name>
Subject: Re: bug#27563: [PATCH v3 2/2] gnu: ghostscript: Write document ID
 only when encrypting.
Date: Fri, 07 Jul 2017 19:45:43 +0200

Danny Milosavljevic <dannym <at> scratchpost.org> skribis:

> I'd vote for an environment variable to disable UUID printing and also
> Time header printing.  That way it would do everything normally in
> regular usage - but when used in packages, it would just not *print*
> the problematic stuff.  No internal state is changed at all by the
> patches.

That sounds like a reasonable approach to me.

I’d make it opt-out to minimize disruption—i.e., the env var would allow
users to disable UUID generation, which would still be enabled as before
otherwise.

Ludo’.

Message #81 received at 27563 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 27563 <at> debbugs.gnu.org
Subject: Re: bug#27563: [PATCH v3 0/2] Make ghostscript reproducible.
Date: Fri, 07 Jul 2017 19:46:47 +0200

Danny Milosavljevic <dannym <at> scratchpost.org> skribis:

>> <https://wiki.debian.org/ReproducibleBuilds/PdfGeneratedByGhostscript>.
>
> Hmm... can you access the patch linked there (under "Solution") ?

It’s 404, but Leo sent a link to the patch on debian.org.

Ludo’.

Message #84 received at 27563 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 27563 <at> debbugs.gnu.org
Subject: Re: bug#27563: [PATCH v3 0/2] Make ghostscript reproducible.
Date: Fri, 07 Jul 2017 19:51:10 +0200

Danny Milosavljevic <dannym <at> scratchpost.org> skribis:

> On Fri, 07 Jul 2017 17:18:15 +0200
> ludo <at> gnu.org (Ludovic Courtès) wrote:
>
>> OK, makes sense.  Maybe we can still have it disabled (or enabled) by
>> environment variable
>
> Sure.  Any suggestions for the name of the environment variable?

For CreationDate/ModDate, I think it should honor SOURCE_DATE_EPOCH as
in
<https://anonscm.debian.org/git/printing/ghostscript.git/tree/debian/patches/2010_add_build_timestamp_setting.patch?id=e2bf3ad7026afe13636d4937430c3fdae7854078>.

For the two UUIDs (and “ID” too?), maybe we can use, say,
GS_GENERATE_UUIDS; if set to 0 or “no” it’s disable, otherwise it’s
enabled.

> Also, where would we set it so the build processes of all the other
> packages actually pick it up?

Eventually we can add it to gnu-build-system.scm, but for now, given
that core-updates is well built, we should add it on a case-by-case
basis.  I don’t think there are that many packages that produce PDFs,
but I could be wrong.

How does that sound?

Thank you,
Ludo’.

Message #87 received at 27563 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 27563 <at> debbugs.gnu.org
Subject: Re: bug#27563: [PATCH v3 0/2] Make ghostscript reproducible.
Date: Fri, 7 Jul 2017 20:20:31 +0200

On Fri, 07 Jul 2017 19:51:10 +0200
ludo <at> gnu.org (Ludovic Courtès) wrote:

> For CreationDate/ModDate, I think it should honor SOURCE_DATE_EPOCH as
> in
> <https://anonscm.debian.org/git/printing/ghostscript.git/tree/debian/patches/2010_add_build_timestamp_setting.patch?id=e2bf3ad7026afe13636d4937430c3fdae7854078>.

Really?  I've been leaving them off, too.  Especially because of this funny comment in the upstream ghostscript:

/* Initialize the IDs allocated at startup. */
void
pdf_initialize_ids(gx_device_pdf * pdev)
{
...
    /*
     * Acrobat Distiller sets CreationDate and ModDate to the current
     * date and time, rather than (for example) %%CreationDate from the
     * PostScript file.  We think this is wrong, but we do the same.
     */
    {
... proceed to set CreationDate and ModDate to the current time.
    }
}

> For the two UUIDs (and “ID” too?), maybe we can use, say,
> GS_GENERATE_UUIDS; if set to 0 or “no” it’s disable, otherwise it’s
> enabled.

That would look like this:

if (!getenv("GS_GENERATE_UUIDS") || strcmp(getenv("GS_GENERATE_UUIDS"), "0") == 0 || strcmp(getenv("GS_GENERATE_UUIDS"), "no") == 0) ...

> > Also, where would we set it so the build processes of all the other
> > packages actually pick it up?  
> 
> Eventually we can add it to gnu-build-system.scm, but for now, given
> that core-updates is well built, we should add it on a case-by-case
> basis.  I don’t think there are that many packages that produce PDFs,
> but I could be wrong.

Okay :)
	

Message #90 received at 27563 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 27563 <at> debbugs.gnu.org, Ludovic Courtès <ludo <at> gnu.org>
Subject: Re: bug#27563: [PATCH v3 2/2] gnu: ghostscript: Write document ID
 only when encrypting.
Date: Sat, 8 Jul 2017 09:50:30 -0400

[Message part 1 (text/plain, inline)]

On Fri, Jul 07, 2017 at 01:24:07PM -0400, Leo Famulari wrote:
> Okay, thank you for explaining this (especially if you already explained
> it! It's hard to join a conversation like this halfway through). I'll
> read your patches carefully later today.

I didn't get around to it :/

I think that if you are Ludo and confident and everything seems to work,
the patches are fine, right? :)

[signature.asc (application/pgp-signature, inline)]

Message #93 received at 27563 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 27563 <at> debbugs.gnu.org
Subject: Re: bug#27563: [PATCH v3 0/2] Make ghostscript reproducible.
Date: Sat, 08 Jul 2017 16:32:02 +0200

Danny Milosavljevic <dannym <at> scratchpost.org> skribis:

> On Fri, 07 Jul 2017 19:51:10 +0200
> ludo <at> gnu.org (Ludovic Courtès) wrote:
>
>> For CreationDate/ModDate, I think it should honor SOURCE_DATE_EPOCH as
>> in
>> <https://anonscm.debian.org/git/printing/ghostscript.git/tree/debian/patches/2010_add_build_timestamp_setting.patch?id=e2bf3ad7026afe13636d4937430c3fdae7854078>.
>
> Really?  I've been leaving them off, too.  Especially because of this funny comment in the upstream ghostscript:
>
> /* Initialize the IDs allocated at startup. */
> void
> pdf_initialize_ids(gx_device_pdf * pdev)
> {
> ...
>     /*
>      * Acrobat Distiller sets CreationDate and ModDate to the current
>      * date and time, rather than (for example) %%CreationDate from the
>      * PostScript file.  We think this is wrong, but we do the same.
>      */
>     {
> ... proceed to set CreationDate and ModDate to the current time.
>     }
> }

I guess they hamper reproducibility if they’re always created?  In that
case, they need to follow SOURCE_DATE_EPOCH; if OTOH they’re only
created in specific cases that don’t matter much, we can leave them.

>> For the two UUIDs (and “ID” too?), maybe we can use, say,
>> GS_GENERATE_UUIDS; if set to 0 or “no” it’s disable, otherwise it’s
>> enabled.
>
> That would look like this:
>
> if (!getenv("GS_GENERATE_UUIDS") || strcmp(getenv("GS_GENERATE_UUIDS"), "0") == 0 || strcmp(getenv("GS_GENERATE_UUIDS"), "no") == 0) ...

Yes.

Thanks!

Ludo’.

Message #96 received at 27563 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: 27563 <at> debbugs.gnu.org
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>
Subject: [PATCH v4 0/3] Make ghostscript reproducible.
Date: Mon, 10 Jul 2017 01:40:12 +0200

Danny Milosavljevic (3):
  gnu: ghostscript: Make "/ID" optional, depending on environment
    variable.
  gnu: ghostscript: Make XMP UUID headers optional, depending on
    environment variable.
  gnu: ghostscript: Make "/CreationDate", "/ModDate" optoinal, depending
    on environment variable.

 gnu/local.mk                                       |  3 ++
 gnu/packages/ghostscript.scm                       | 10 ++---
 .../ghostscript-no-header-creationdate.patch       | 16 +++++++
 .../patches/ghostscript-no-header-id.patch         | 49 ++++++++++++++++++++++
 .../patches/ghostscript-no-header-uuid.patch       | 43 +++++++++++++++++++
 5 files changed, 116 insertions(+), 5 deletions(-)
 create mode 100644 gnu/packages/patches/ghostscript-no-header-creationdate.patch
 create mode 100644 gnu/packages/patches/ghostscript-no-header-id.patch
 create mode 100644 gnu/packages/patches/ghostscript-no-header-uuid.patch

Message #99 received at 27563 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: 27563 <at> debbugs.gnu.org
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>
Subject: [PATCH v4 2/3] gnu: ghostscript: Make XMP UUID headers optional,
 depending on environment variable.
Date: Mon, 10 Jul 2017 01:46:43 +0200

* gnu/packages/patches/ghostscript-no-header-uuid.patch: New file.
* gnu/local.mk (dist_patch_DATA): Add it.
* gnu/packages/ghostscript.scm (ghostscript): Use it.
---
 gnu/local.mk                                       |  1 +
 gnu/packages/ghostscript.scm                       |  3 +-
 .../patches/ghostscript-no-header-uuid.patch       | 43 ++++++++++++++++++++++
 3 files changed, 46 insertions(+), 1 deletion(-)
 create mode 100644 gnu/packages/patches/ghostscript-no-header-uuid.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index 8cbded44e..9cf4648aa 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -620,6 +620,7 @@ dist_patch_DATA =						\
   %D%/packages/patches/ghc-dont-pass-linker-flags-via-response-files.patch	\
   %D%/packages/patches/ghostscript-CVE-2017-8291.patch		\
   %D%/packages/patches/ghostscript-no-header-id.patch		\
+  %D%/packages/patches/ghostscript-no-header-uuid.patch		\
   %D%/packages/patches/ghostscript-runpath.patch		\
   %D%/packages/patches/glib-networking-ssl-cert-file.patch	\
   %D%/packages/patches/glib-tests-timer.patch			\
diff --git a/gnu/packages/ghostscript.scm b/gnu/packages/ghostscript.scm
index af565f3e3..3a84db3e4 100644
--- a/gnu/packages/ghostscript.scm
+++ b/gnu/packages/ghostscript.scm
@@ -144,7 +144,8 @@ printing, and psresize, for adjusting page sizes.")
           "0lyhjcrkmd5fcmh8h56bs4xr9k4jasmikv5vsix1hd4ai0ad1q9b"))
         (patches (search-patches "ghostscript-runpath.patch"
                                  "ghostscript-CVE-2017-8291.patch"
-                                 "ghostscript-no-header-id.patch"))
+                                 "ghostscript-no-header-id.patch"
+                                 "ghostscript-no-header-uuid.patch"))
         (modules '((guix build utils)))
         (snippet
           ;; Remove bundled libraries. The bundled OpenJPEG is a patched fork so
diff --git a/gnu/packages/patches/ghostscript-no-header-uuid.patch b/gnu/packages/patches/ghostscript-no-header-uuid.patch
new file mode 100644
index 000000000..53bef90f0
--- /dev/null
+++ b/gnu/packages/patches/ghostscript-no-header-uuid.patch
@@ -0,0 +1,43 @@
+This patch makes the UUIDs in the XMP header optional, depending on the
+setting of the environment variable GS_GENERATE_UUIDS.
+
+diff -ur orig/gnu-ghostscript-9.14.0/devices/vector/gdevpdfe.c aa/gnu-ghostscript-9.14.0/devices/vector/gdevpdfe.c
+--- orig/gnu-ghostscript-9.14.0/devices/vector/gdevpdfe.c	2017-07-09 23:30:28.960479189 +0200
++++ gnu-ghostscript-9.14.0/devices/vector/gdevpdfe.c	2017-07-10 01:04:12.252478276 +0200
+@@ -617,7 +617,7 @@
+         return code;
+ 
+     /* PDF/A XMP reference recommends setting UUID to empty. If not empty must be a URI */
+-    if (pdev->PDFA != 0)
++    if (pdev->PDFA != 0 || (getenv("GS_GENERATE_UUIDS") && (strcmp(getenv("GS_GENERATE_UUIDS"), "0") == 0 || strcmp(getenv("GS_GENERATE_UUIDS"), "no") == 0)))
+         instance_uuid[0] = 0x00;
+ 
+     cre_date_time_len = pdf_get_docinfo_item(pdev, "/CreationDate", cre_date_time, sizeof(cre_date_time));
+@@ -719,15 +719,18 @@
+             pdf_xml_tag_close(s, "rdf:Description");
+             pdf_xml_newline(s);
+ 
+-            pdf_xml_tag_open_beg(s, "rdf:Description");
+-            pdf_xml_attribute_name(s, "rdf:about");
+-            pdf_xml_attribute_value(s, instance_uuid);
+-            pdf_xml_attribute_name(s, "xmlns:xapMM");
+-            pdf_xml_attribute_value(s, "http://ns.adobe.com/xap/1.0/mm/");
+-            pdf_xml_attribute_name(s, "xapMM:DocumentID");
+-            pdf_xml_attribute_value(s, document_uuid);
+-            pdf_xml_tag_end_empty(s);
+-            pdf_xml_newline(s);
++            if (!getenv("GS_GENERATE_UUIDS") || (strcmp(getenv("GS_GENERATE_UUIDS"), "0") != 0 && strcmp(getenv("GS_GENERATE_UUIDS"), "no") != 0))
++            {
++                pdf_xml_tag_open_beg(s, "rdf:Description");
++                pdf_xml_attribute_name(s, "rdf:about");
++                pdf_xml_attribute_value(s, instance_uuid);
++                pdf_xml_attribute_name(s, "xmlns:xapMM");
++                pdf_xml_attribute_value(s, "http://ns.adobe.com/xap/1.0/mm/");
++                pdf_xml_attribute_name(s, "xapMM:DocumentID");
++                pdf_xml_attribute_value(s, document_uuid);
++                pdf_xml_tag_end_empty(s);
++                pdf_xml_newline(s);
++            }
+ 
+             pdf_xml_tag_open_beg(s, "rdf:Description");
+             pdf_xml_attribute_name(s, "rdf:about");

Message #102 received at 27563 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: 27563 <at> debbugs.gnu.org
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>
Subject: [PATCH v4 1/3] gnu: ghostscript: Make "/ID" optional,
 depending on environment variable.
Date: Mon, 10 Jul 2017 01:46:42 +0200

* gnu/packages/patches/ghostscript-no-header-id.patch: New file.
* gnu/local.mk (dist_patch_DATA): Add it
* gnu/packages/ghostscript.scm (ghostscript): Use it.
---
 gnu/local.mk                                       |  1 +
 gnu/packages/ghostscript.scm                       |  9 +++-
 .../patches/ghostscript-no-header-id.patch         | 49 ++++++++++++++++++++++
 3 files changed, 57 insertions(+), 2 deletions(-)
 create mode 100644 gnu/packages/patches/ghostscript-no-header-id.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index 0fe6cdc39..8cbded44e 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -619,6 +619,7 @@ dist_patch_DATA =						\
   %D%/packages/patches/gettext-gnulib-multi-core.patch          \
   %D%/packages/patches/ghc-dont-pass-linker-flags-via-response-files.patch	\
   %D%/packages/patches/ghostscript-CVE-2017-8291.patch		\
+  %D%/packages/patches/ghostscript-no-header-id.patch		\
   %D%/packages/patches/ghostscript-runpath.patch		\
   %D%/packages/patches/glib-networking-ssl-cert-file.patch	\
   %D%/packages/patches/glib-tests-timer.patch			\
diff --git a/gnu/packages/ghostscript.scm b/gnu/packages/ghostscript.scm
index b9ba1c081..af565f3e3 100644
--- a/gnu/packages/ghostscript.scm
+++ b/gnu/packages/ghostscript.scm
@@ -143,7 +143,8 @@ printing, and psresize, for adjusting page sizes.")
          (base32
           "0lyhjcrkmd5fcmh8h56bs4xr9k4jasmikv5vsix1hd4ai0ad1q9b"))
         (patches (search-patches "ghostscript-runpath.patch"
-                                 "ghostscript-CVE-2017-8291.patch"))
+                                 "ghostscript-CVE-2017-8291.patch"
+                                 "ghostscript-no-header-id.patch"))
         (modules '((guix build utils)))
         (snippet
           ;; Remove bundled libraries. The bundled OpenJPEG is a patched fork so
@@ -155,7 +156,11 @@ printing, and psresize, for adjusting page sizes.")
                                                 "tiff" "zlib"))
             ;; Get rid of timestamps (remove /CreationDate and /ModDate).
             (substitute* "devices/vector/gdevpdf.c"
-             ((", but we do the same") "*/ if (0) /*"))))))
+             ((", but we do the same")
+              (string-append "*/ "
+                             "if (!getenv(\"GS_GENERATE_UUIDS\") || "
+                             "(strcmp(getenv(\"GS_GENERATE_UUIDS\"), \"0\") != 0 && "
+                             "strcmp(getenv(\"GS_GENERATE_UUIDS\"), \"no\") != 0)) /*")))))))
     (build-system gnu-build-system)
     (outputs '("out" "doc"))                  ;19 MiB of HTML/PS doc + examples
     (arguments
diff --git a/gnu/packages/patches/ghostscript-no-header-id.patch b/gnu/packages/patches/ghostscript-no-header-id.patch
new file mode 100644
index 000000000..1e49921ad
--- /dev/null
+++ b/gnu/packages/patches/ghostscript-no-header-id.patch
@@ -0,0 +1,49 @@
+diff -ur orig/gnu-ghostscript-9.14.0/devices/vector/gdevpdf.c gnu-ghostscript-9.14.0/devices/vector/gdevpdf.c
+--- orig/gnu-ghostscript-9.14.0/devices/vector/gdevpdf.c	2017-07-09 23:30:28.960479189 +0200
++++ gnu-ghostscript-9.14.0/devices/vector/gdevpdf.c	2017-07-09 23:34:34.306524488 +0200
+@@ -1580,8 +1580,11 @@
+      * +1 for the linearisation dict and +1 for the primary hint stream.
+      */
+     linear_params->FirsttrailerOffset = gp_ftell_64(linear_params->Lin_File.file);
+-    gs_sprintf(LDict, "\ntrailer\n<</Size %ld/Info %d 0 R/Root %d 0 R/ID[%s%s]/Prev %d>>\nstartxref\r\n0\n%%%%EOF\n        \n",
+-        linear_params->LastResource + 3, pdev->ResourceUsage[linear_params->Info_id].NewObjectNumber, pdev->ResourceUsage[linear_params->Catalog_id].NewObjectNumber, fileID, fileID, 0);
++    gs_sprintf(LDict, "\ntrailer\n<</Size %ld/Info %d 0 R/Root %d 0 R",
++        linear_params->LastResource + 3, pdev->ResourceUsage[linear_params->Info_id].NewObjectNumber, pdev->ResourceUsage[linear_params->Catalog_id].NewObjectNumber);
++    if (pdev->OwnerPassword.size > 0 || !(!getenv("GS_GENERATE_UUIDS") || (strcmp(getenv("GS_GENERATE_UUIDS"), "0") != 0 && strcmp(getenv("GS_GENERATE_UUIDS"), "no") != 0))) /* ID is mandatory when encrypting */
++        gs_sprintf(LDict, "/ID[%s%s]", fileID, fileID);
++    gs_sprintf(LDict, "/Prev %d>>\nstartxref\r\n0\n%%%%EOF\n        \n", 0);
+     fwrite(LDict, strlen(LDict), 1, linear_params->Lin_File.file);
+ 
+     /* Write document catalog (Part 4) */
+@@ -2102,8 +2105,11 @@
+      * in the missing values.
+      */
+     code = gp_fseek_64(linear_params->sfile, linear_params->FirsttrailerOffset, SEEK_SET);
+-    gs_sprintf(LDict, "\ntrailer\n<</Size %ld/Info %d 0 R/Root %d 0 R/ID[%s%s]/Prev %"PRId64">>\nstartxref\r\n0\n%%%%EOF\n",
+-        linear_params->LastResource + 3, pdev->ResourceUsage[linear_params->Info_id].NewObjectNumber, pdev->ResourceUsage[linear_params->Catalog_id].NewObjectNumber, fileID, fileID, mainxref);
++    gs_sprintf(LDict, "\ntrailer\n<</Size %ld/Info %d 0 R/Root %d 0 R",
++        linear_params->LastResource + 3, pdev->ResourceUsage[linear_params->Info_id].NewObjectNumber, pdev->ResourceUsage[linear_params->Catalog_id].NewObjectNumber);
++    if (pdev->OwnerPassword.size > 0 || !(!getenv("GS_GENERATE_UUIDS") || (strcmp(getenv("GS_GENERATE_UUIDS"), "0") != 0 || strcmp(getenv("GS_GENERATE_UUIDS"), "no") != 0))) /* ID is mandatory when encrypting */
++        gs_sprintf(LDict, "/ID[%s%s]", fileID, fileID);
++    gs_sprintf(LDict, "/Prev %"PRId64">>\nstartxref\r\n0\n%%%%EOF\n", mainxref);
+     fwrite(LDict, strlen(LDict), 1, linear_params->sfile);
+ 
+     code = gp_fseek_64(linear_params->sfile, pdev->ResourceUsage[HintStreamObj].LinearisedOffset, SEEK_SET);
+@@ -2674,10 +2680,12 @@
+             stream_puts(s, "trailer\n");
+             pprintld3(s, "<< /Size %ld /Root %ld 0 R /Info %ld 0 R\n",
+                   pdev->next_id, Catalog_id, Info_id);
+-            stream_puts(s, "/ID [");
+-            psdf_write_string(pdev->strm, pdev->fileID, sizeof(pdev->fileID), 0);
+-            psdf_write_string(pdev->strm, pdev->fileID, sizeof(pdev->fileID), 0);
+-            stream_puts(s, "]\n");
++            if (pdev->OwnerPassword.size > 0 || !(!getenv("GS_GENERATE_UUIDS") || (strcmp(getenv("GS_GENERATE_UUIDS"), "0") != 0 || strcmp(getenv("GS_GENERATE_UUIDS"), "no") != 0))) { /* ID is mandatory when encrypting */
++                stream_puts(s, "/ID [");
++                psdf_write_string(pdev->strm, pdev->fileID, sizeof(pdev->fileID), 0);
++                psdf_write_string(pdev->strm, pdev->fileID, sizeof(pdev->fileID), 0);
++                stream_puts(s, "]\n");
++            }
+             if (pdev->OwnerPassword.size > 0) {
+                 pprintld1(s, "/Encrypt %ld 0 R ", Encrypt_id);
+             }
+Nur in gnu-ghostscript-9.14.0/devices/vector: gdevpdf.c.orig.

Message #105 received at 27563 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: 27563 <at> debbugs.gnu.org
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>
Subject: [PATCH v4 3/3] gnu: ghostscript: Make "/CreationDate",
 "/ModDate" optoinal, depending on environment variable.
Date: Mon, 10 Jul 2017 01:46:44 +0200

* gnu/packages/patches/ghostscript-no-header-creationdate.patch: New file.
* gnu/local.mk (dist_patch_DATA): Add it.
* gnu/packages/ghostscript.scm (ghostscript): Use it.
---
 gnu/local.mk                                             |  1 +
 gnu/packages/ghostscript.scm                             | 10 ++--------
 .../patches/ghostscript-no-header-creationdate.patch     | 16 ++++++++++++++++
 3 files changed, 19 insertions(+), 8 deletions(-)
 create mode 100644 gnu/packages/patches/ghostscript-no-header-creationdate.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index 9cf4648aa..0880de038 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -621,6 +621,7 @@ dist_patch_DATA =						\
   %D%/packages/patches/ghostscript-CVE-2017-8291.patch		\
   %D%/packages/patches/ghostscript-no-header-id.patch		\
   %D%/packages/patches/ghostscript-no-header-uuid.patch		\
+  %D%/packages/patches/ghostscript-no-header-creationdate.patch \
   %D%/packages/patches/ghostscript-runpath.patch		\
   %D%/packages/patches/glib-networking-ssl-cert-file.patch	\
   %D%/packages/patches/glib-tests-timer.patch			\
diff --git a/gnu/packages/ghostscript.scm b/gnu/packages/ghostscript.scm
index 3a84db3e4..b15b032a7 100644
--- a/gnu/packages/ghostscript.scm
+++ b/gnu/packages/ghostscript.scm
@@ -144,6 +144,7 @@ printing, and psresize, for adjusting page sizes.")
           "0lyhjcrkmd5fcmh8h56bs4xr9k4jasmikv5vsix1hd4ai0ad1q9b"))
         (patches (search-patches "ghostscript-runpath.patch"
                                  "ghostscript-CVE-2017-8291.patch"
+                                 "ghostscript-no-header-creationdate.patch"
                                  "ghostscript-no-header-id.patch"
                                  "ghostscript-no-header-uuid.patch"))
         (modules '((guix build utils)))
@@ -154,14 +155,7 @@ printing, and psresize, for adjusting page sizes.")
          '(begin
             (for-each delete-file-recursively '("freetype" "jbig2dec" "jpeg"
                                                 "lcms2" "libpng"
-                                                "tiff" "zlib"))
-            ;; Get rid of timestamps (remove /CreationDate and /ModDate).
-            (substitute* "devices/vector/gdevpdf.c"
-             ((", but we do the same")
-              (string-append "*/ "
-                             "if (!getenv(\"GS_GENERATE_UUIDS\") || "
-                             "(strcmp(getenv(\"GS_GENERATE_UUIDS\"), \"0\") != 0 && "
-                             "strcmp(getenv(\"GS_GENERATE_UUIDS\"), \"no\") != 0)) /*")))))))
+                                                "tiff" "zlib"))))))
     (build-system gnu-build-system)
     (outputs '("out" "doc"))                  ;19 MiB of HTML/PS doc + examples
     (arguments
diff --git a/gnu/packages/patches/ghostscript-no-header-creationdate.patch b/gnu/packages/patches/ghostscript-no-header-creationdate.patch
new file mode 100644
index 000000000..2601938b7
--- /dev/null
+++ b/gnu/packages/patches/ghostscript-no-header-creationdate.patch
@@ -0,0 +1,16 @@
+This patch makes emission of /CreationDate and /ModDate headers optional.
+
+diff --git a/orig/gnu-ghostscript-9.14.0/devices/vector/gdevpdf.c b/bb/gnu-ghostscript-9.14.0/devices/vector/gdevpdf.c
+index 0fb067e..b342e2c 100644
+--- orig/gnu-ghostscript-9.14.0/devices/vector/gdevpdf.c
++++ gnu-ghostscript-9.14.0/devices/vector/gdevpdf.c
+@@ -305,6 +305,9 @@ pdf_initialize_ids(gx_device_pdf * pdev)
+      * date and time, rather than (for example) %%CreationDate from the
+      * PostScript file.  We think this is wrong, but we do the same.
+      */
++    if (!getenv("GS_GENERATE_UUIDS") ||
++        (strcmp(getenv("GS_GENERATE_UUIDS"), "0") != 0 &&
++         strcmp(getenv("GS_GENERATE_UUIDS"), "no") != 0))
+     {
+         struct tm tms;
+         time_t t;

Message #108 received at 27563 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 27563 <at> debbugs.gnu.org
Subject: Re: bug#27563: [PATCH v4 1/3] gnu: ghostscript: Make "/ID" optional,
 depending on environment variable.
Date: Mon, 10 Jul 2017 17:21:07 +0200

Danny Milosavljevic <dannym <at> scratchpost.org> skribis:

> * gnu/packages/patches/ghostscript-no-header-id.patch: New file.
> * gnu/local.mk (dist_patch_DATA): Add it
> * gnu/packages/ghostscript.scm (ghostscript): Use it.

[...]

> diff --git a/gnu/packages/patches/ghostscript-no-header-id.patch b/gnu/packages/patches/ghostscript-no-header-id.patch
> new file mode 100644
> index 000000000..1e49921ad
> --- /dev/null
> +++ b/gnu/packages/patches/ghostscript-no-header-id.patch

Please include a description of what the patch does and what its
upstream status is (you can write that it was not submitted upstream but
that similar patches were discussed, linking to this bug and upstream
URLs.)

> +Nur in gnu-ghostscript-9.14.0/devices/vector: gdevpdf.c.orig.

Extra line.   :-)

I haven’t tested the patch, but if you can confirm that ps2pdf still
generates valid PDF files after this change, both with and without
UUIDs, then that’s fine with me.

Thank you!

Ludo’.

Message #111 received at 27563 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 27563 <at> debbugs.gnu.org
Subject: Re: bug#27563: [PATCH v4 2/3] gnu: ghostscript: Make XMP UUID headers
 optional, depending on environment variable.
Date: Mon, 10 Jul 2017 17:23:20 +0200

Danny Milosavljevic <dannym <at> scratchpost.org> skribis:

> * gnu/packages/patches/ghostscript-no-header-uuid.patch: New file.
> * gnu/local.mk (dist_patch_DATA): Add it.
> * gnu/packages/ghostscript.scm (ghostscript): Use it.

[...]

> --- /dev/null
> +++ b/gnu/packages/patches/ghostscript-no-header-uuid.patch
> @@ -0,0 +1,43 @@
> +This patch makes the UUIDs in the XMP header optional, depending on the
> +setting of the environment variable GS_GENERATE_UUIDS.
> +
> +diff -ur orig/gnu-ghostscript-9.14.0/devices/vector/gdevpdfe.c aa/gnu-ghostscript-9.14.0/devices/vector/gdevpdfe.

Does this also apply to ‘core-updates’?

> +--- orig/gnu-ghostscript-9.14.0/devices/vector/gdevpdfe.c	2017-07-09 23:30:28.960479189 +0200
> ++++ gnu-ghostscript-9.14.0/devices/vector/gdevpdfe.c	2017-07-10 01:04:12.252478276 +0200
> +@@ -617,7 +617,7 @@
> +         return code;
> + 
> +     /* PDF/A XMP reference recommends setting UUID to empty. If not empty must be a URI */
> +-    if (pdev->PDFA != 0)
> ++    if (pdev->PDFA != 0 || (getenv("GS_GENERATE_UUIDS") && (strcmp(getenv("GS_GENERATE_UUIDS"), "0") == 0 || strcmp(getenv("GS_GENERATE_UUIDS"), "no") == 0)))

Rather ‘strcasecmp’ for the last one.

If the resulting ps2pdf works, OK for ‘core-updates’!

Ludo’.

Message #114 received at 27563 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 27563 <at> debbugs.gnu.org
Subject: Re: bug#27563: [PATCH v4 3/3] gnu: ghostscript: Make "/CreationDate",
 "/ModDate" optoinal, depending on environment variable.
Date: Mon, 10 Jul 2017 17:25:14 +0200

Danny Milosavljevic <dannym <at> scratchpost.org> skribis:

> * gnu/packages/patches/ghostscript-no-header-creationdate.patch: New file.
> * gnu/local.mk (dist_patch_DATA): Add it.
> * gnu/packages/ghostscript.scm (ghostscript): Use it.
> ---
>  gnu/local.mk                                             |  1 +
>  gnu/packages/ghostscript.scm                             | 10 ++--------
>  .../patches/ghostscript-no-header-creationdate.patch     | 16 ++++++++++++++++
>  3 files changed, 19 insertions(+), 8 deletions(-)
>  create mode 100644 gnu/packages/patches/ghostscript-no-header-creationdate.patch
>
> diff --git a/gnu/local.mk b/gnu/local.mk
> index 9cf4648aa..0880de038 100644
> --- a/gnu/local.mk
> +++ b/gnu/local.mk
> @@ -621,6 +621,7 @@ dist_patch_DATA =						\
>    %D%/packages/patches/ghostscript-CVE-2017-8291.patch		\
>    %D%/packages/patches/ghostscript-no-header-id.patch		\
>    %D%/packages/patches/ghostscript-no-header-uuid.patch		\
> +  %D%/packages/patches/ghostscript-no-header-creationdate.patch \
>    %D%/packages/patches/ghostscript-runpath.patch		\
>    %D%/packages/patches/glib-networking-ssl-cert-file.patch	\
>    %D%/packages/patches/glib-tests-timer.patch			\
> diff --git a/gnu/packages/ghostscript.scm b/gnu/packages/ghostscript.scm
> index 3a84db3e4..b15b032a7 100644
> --- a/gnu/packages/ghostscript.scm
> +++ b/gnu/packages/ghostscript.scm
> @@ -144,6 +144,7 @@ printing, and psresize, for adjusting page sizes.")
>            "0lyhjcrkmd5fcmh8h56bs4xr9k4jasmikv5vsix1hd4ai0ad1q9b"))
>          (patches (search-patches "ghostscript-runpath.patch"
>                                   "ghostscript-CVE-2017-8291.patch"
> +                                 "ghostscript-no-header-creationdate.patch"
>                                   "ghostscript-no-header-id.patch"
>                                   "ghostscript-no-header-uuid.patch"))
>          (modules '((guix build utils)))
> @@ -154,14 +155,7 @@ printing, and psresize, for adjusting page sizes.")
>           '(begin
>              (for-each delete-file-recursively '("freetype" "jbig2dec" "jpeg"
>                                                  "lcms2" "libpng"
> -                                                "tiff" "zlib"))
> -            ;; Get rid of timestamps (remove /CreationDate and /ModDate).
> -            (substitute* "devices/vector/gdevpdf.c"
> -             ((", but we do the same")
> -              (string-append "*/ "
> -                             "if (!getenv(\"GS_GENERATE_UUIDS\") || "
> -                             "(strcmp(getenv(\"GS_GENERATE_UUIDS\"), \"0\") != 0 && "
> -                             "strcmp(getenv(\"GS_GENERATE_UUIDS\"), \"no\") != 0)) /*")))))))
> +                                                "tiff" "zlib"))))))

Hmm, what is this patch against?  Was it generated against a previous
WIP tree?

Ludo’.

Message #119 received at 27563-done <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: 27563-done <at> debbugs.gnu.org
Subject: Re: [PATCH v4 0/3] Make ghostscript reproducible.
Date: Mon, 10 Jul 2017 18:35:48 +0200

I tested it, with bug# 27593 it works and creates a reproducible groff.

So I pushed this set to core-updates; closing this bug report.

Phiew, finally almost done.  There's one more patch for groff at bug# 27593 in order to set the environment variable GS_GENERATE_UUIDS to "0".

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.