GNU bug report logs - #29700
Java packages need to retain references

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 29700 in the body.
You can then email your comments to 29700 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ricardo Wurmus <rekado <at> elephly.net>
To: bug-guix <at> gnu.org
Subject: Java packages need to retain references
Date: Wed, 13 Dec 2017 23:48:24 +0100

I installed the “axoloti-patcher” package today after “guix gc” and
noticed that not all dependencies are available at runtime.  Upon
running the “Axoloti” command I got this error:

    Exception in thread "AWT-EventQueue-0" java.lang.UnsatisfiedLinkError: Can't load library: /gnu/store/nrx5nmwynmsl7aqxdpgwpsm8ff9b0h4d-libusb4java-0-1.396d642a5/lib/libusb4java.so

The store item does not exist.

The problem here is that jar archives are compressed and thus any
references they contain are thus hidden from the scanner.

I don’t know if it is feasible to disable compression for jar files, or
if we should install a “.guix-dependencies” file that contains plain
text references.

I’ll investigate disabling compression first.  Looks like we can do this
in the strip-jar-timestamps phase where we use “zip” to repack the jar.
Adding “-0” to the arguments should be enough to make this work.

I’ll send a patch soon.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Message #8 received at 29700 <at> debbugs.gnu.org (full text, mbox):

From: Ricardo Wurmus <rekado <at> elephly.net>
To: 29700 <at> debbugs.gnu.org
Cc: Ricardo Wurmus <rekado <at> elephly.net>
Subject: [PATCH] guix: ant-build-system: Do not compress jars.
Date: Thu, 14 Dec 2017 00:09:31 +0100

Fixes <https://bugs.gnu.org/29700>.

* guix/build/ant-build-system.scm (strip-jar-timestamps): Do not compress jar
when repacking.
---
 guix/build/ant-build-system.scm | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/guix/build/ant-build-system.scm b/guix/build/ant-build-system.scm
index a440daf05..6ce813a00 100644
--- a/guix/build/ant-build-system.scm
+++ b/guix/build/ant-build-system.scm
@@ -197,9 +197,12 @@ repack them.  This is necessary to ensure that archives are reproducible."
            ;; first.
            (with-directory-excursion dir
              (let* ((files (find-files "." ".*" #:directories? #t))
+                    ;; To ensure that the reference scanner can detect all
+                    ;; store references in the jars we disable compression
+                    ;; with the "-0" option.
                     (command (if (file-exists? manifest)
-                                 `("zip" "-X" ,jar ,manifest ,@files)
-                                 `("zip" "-X" ,jar ,@files))))
+                                 `("zip" "-0" "-X" ,jar ,manifest ,@files)
+                                 `("zip" "-0" "-X" ,jar ,@files))))
                (unless (zero? (apply system* command))
                  (error "'zip' failed"))))
            (utime jar 0 0)
-- 
2.15.0

Message #11 received at 29700 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: Ricardo Wurmus <rekado <at> elephly.net>
Cc: 29700 <at> debbugs.gnu.org
Subject: Re: bug#29700: [PATCH] guix: ant-build-system: Do not compress jars.
Date: Thu, 14 Dec 2017 09:23:54 +0100

LGTM!

Message #14 received at 29700 <at> debbugs.gnu.org (full text, mbox):

From: Ricardo Wurmus <rekado <at> elephly.net>
To: Gábor Boskovits <boskovits <at> gmail.com>
Cc: 29700 <at> debbugs.gnu.org
Subject: Re: bug#29700: Java packages need to retain references
Date: Thu, 14 Dec 2017 23:22:21 +0100

Gábor Boskovits <boskovits <at> gmail.com> writes:

> Would it be enough if we added the uncompressed manifest?
> Is there any drawback to that?

I will test that.  I’m not sure if the manifest is really sufficient.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Message #21 received at 29700-done <at> debbugs.gnu.org (full text, mbox):

From: Ricardo Wurmus <rekado <at> elephly.net>
To: Gábor Boskovits <boskovits <at> gmail.com>
Cc: 29700-done <at> debbugs.gnu.org
Subject: Re: bug#29700: Java packages need to retain references
Date: Thu, 14 Dec 2017 23:59:22 +0100

Hi Gábor,

The manifest file would not be sufficient.  Take the example of
“java-usb4java”.  It provides a file “share/java/usb4java.jar”, which
contains this manifest:

--8<---------------cut here---------------start------------->8---
Manifest-Version: 1.0
Created-By: 1.7.0_161 (Oracle Corporation)

--8<---------------cut here---------------end--------------->8---

When you open the Loader.class file that is included in the archive you
will notice that it contains this string:

    /gnu/store/iwjqaqr98daqbgwswqi5fi4w891bqh7d-libusb4java-0-1.396d642a5/lib/libusb4java.so

This reference is currently dropped because it cannot be detected in a
compressed archive.  The reference is not contained in the manifest, and
at build time we could only guess which references should be added to
the manifest.

Unless we extend the reference scanner to deal with compressed files I
think that disabling compression in jars is the best approach.

I’ll push this to master.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Message #29 received at 29700-done <at> debbugs.gnu.org (full text, mbox):

From: Gábor Boskovits <boskovits <at> gmail.com>
To: Ricardo Wurmus <rekado <at> elephly.net>
Cc: 29700-done <at> debbugs.gnu.org
Subject: Re: bug#29700: Java packages need to retain references
Date: Fri, 15 Dec 2017 07:44:36 +0100

[Message part 1 (text/plain, inline)]

Do we know the impact of disabling compression?
I don't think it is really a big problem.
You are right with this, I'm still a bit stuck in another mindset, where we
usually know, that we have only one version of a package...
If we later decide that compression is desired we can still go with a text
file with references.
Anyway, do you think that extending the reference scanner would be a good
idea?


2017-12-14 23:59 GMT+01:00 Ricardo Wurmus <rekado <at> elephly.net>:

> Hi Gábor,
>
> The manifest file would not be sufficient.  Take the example of
> “java-usb4java”.  It provides a file “share/java/usb4java.jar”, which
> contains this manifest:
>
> --8<---------------cut here---------------start------------->8---
> Manifest-Version: 1.0
> Created-By: 1.7.0_161 (Oracle Corporation)
>
> --8<---------------cut here---------------end--------------->8---
>
> When you open the Loader.class file that is included in the archive you
> will notice that it contains this string:
>
>     /gnu/store/iwjqaqr98daqbgwswqi5fi4w891bqh7d-libusb4java-0-1.396d642a5/
> lib/libusb4java.so
>
> This reference is currently dropped because it cannot be detected in a
> compressed archive.  The reference is not contained in the manifest, and
> at build time we could only guess which references should be added to
> the manifest.
>
> Unless we extend the reference scanner to deal with compressed files I
> think that disabling compression in jars is the best approach.
>
> I’ll push this to master.
>
> --
> Ricardo
>
> GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
> https://elephly.net
>
>
>

[Message part 2 (text/html, inline)]

Message #32 received at 29700-done <at> debbugs.gnu.org (full text, mbox):

From: Ricardo Wurmus <rekado <at> elephly.net>
To: Gábor Boskovits <boskovits <at> gmail.com>
Cc: 29700-done <at> debbugs.gnu.org
Subject: Re: bug#29700: Java packages need to retain references
Date: Fri, 15 Dec 2017 08:49:40 +0100

Hi Gábor,

> If we later decide that compression is desired we can still go with a text
> file with references.

This wouldn’t always be feasible, because it would require *manual*
reference tracking.  The packager would need to know what references
need to be retained and then ensure that they are part of that text
file.

> Anyway, do you think that extending the reference scanner would be a good
> idea?

We had the same problem with gzipped R files and zipped Python things.
For the R files a more advanced reference scanner would not work,
because the files just include a gzipped section; it’s not the whole
file that’s simply gzipped.  Detecting sections is too hard.

But for Python and Jars it might just be enough.

[PS: please don’t top-post.]

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Message #35 received at 29700 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Ricardo Wurmus <rekado <at> elephly.net>
Cc: 29700 <at> debbugs.gnu.org
Subject: Re: bug#29700: Java packages need to retain references
Date: Fri, 15 Dec 2017 15:55:04 +0100

Ricardo Wurmus <rekado <at> elephly.net> skribis:

> I installed the “axoloti-patcher” package today after “guix gc” and
> noticed that not all dependencies are available at runtime.  Upon
> running the “Axoloti” command I got this error:
>
>     Exception in thread "AWT-EventQueue-0" java.lang.UnsatisfiedLinkError: Can't load library: /gnu/store/nrx5nmwynmsl7aqxdpgwpsm8ff9b0h4d-libusb4java-0-1.396d642a5/lib/libusb4java.so
>
> The store item does not exist.
>
> The problem here is that jar archives are compressed and thus any
> references they contain are thus hidden from the scanner.
>
> I don’t know if it is feasible to disable compression for jar files, or
> if we should install a “.guix-dependencies” file that contains plain
> text references.

Disabling compression was the right choice because that also allows
grafts to work correctly.

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.