GNU bug report logs - #31719
icedtea-3 binaries contain references to icedtea-2

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 31719 in the body.
You can then email your comments to 31719 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ricardo Wurmus <ricardo.wurmus <at> mdc-berlin.de>
To: <bug-guix <at> gnu.org>
Cc: Gábor Boskovits <boskovits <at> gmail.com>
Subject: icedtea-3 binaries contain references to icedtea-2
Date: Tue, 5 Jun 2018 12:47:45 +0200

Many binaries of the icedtea-3 package contain references to icedtea-2,
which was used to build it.  Because of these references users of
icedtea-3 have to *also* download icedtea-2.  And because icedtea-2
contains references to icedtea-1, they *also* need to download that
version.

It is possible that icedtea-1 contains references to the bootstrap JDK,
which will then also have to be downloaded.

We should figure out a way to remove all of these references.

-- 
Ricardo

Message #8 received at 31719 <at> debbugs.gnu.org (full text, mbox):

From: Gábor Boskovits <boskovits <at> gmail.com>
To: 31719 <at> debbugs.gnu.org, Ricardo Wurmus <rekado <at> elephly.net>
Subject: icedtea-3 binaries contain references to icedtea-2
Date: Tue, 5 Jun 2018 13:49:12 +0200

[Message part 1 (text/plain, inline)]

Do you think we can do something like rebuild icedtea3 with icedtea3, and
then rewrite the references? I guess this way the dependency loop could be
broken, as we could detach the last icedtea3 build from the packages used
for bootstrapping.
We could even make on more rebuilding, if necessary. WDYT?

[Message part 2 (text/html, inline)]

Message #11 received at 31719 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Gábor Boskovits <boskovits <at> gmail.com>
Cc: Ricardo Wurmus <rekado <at> elephly.net>, 31719 <at> debbugs.gnu.org
Subject: Re: bug#31719: icedtea-3 binaries contain references to icedtea-2
Date: Tue, 5 Jun 2018 08:45:03 -0400

[Message part 1 (text/plain, inline)]

On Tue, Jun 05, 2018 at 01:49:12PM +0200, Gábor Boskovits wrote:
> Do you think we can do something like rebuild icedtea3 with icedtea3, and
> then rewrite the references? I guess this way the dependency loop could be
> broken, as we could detach the last icedtea3 build from the packages used
> for bootstrapping.

If the references are not important (that is, if they are not used), we
could also rewrite them to something that doesn't exist. Currently the
go-build-system does this so that Go executables don't refer to the Go
compiler, which is very large.

[signature.asc (application/pgp-signature, inline)]

Message #14 received at 31719 <at> debbugs.gnu.org (full text, mbox):

From: Gábor Boskovits <boskovits <at> gmail.com>
To: Leo Famulari <leo <at> famulari.name>
Cc: Ricardo Wurmus <rekado <at> elephly.net>, 31719 <at> debbugs.gnu.org
Subject: Re: bug#31719: icedtea-3 binaries contain references to icedtea-2
Date: Tue, 5 Jun 2018 16:45:57 +0200

[Message part 1 (text/plain, inline)]

2018-06-05 14:45 GMT+02:00 Leo Famulari <leo <at> famulari.name>:

> On Tue, Jun 05, 2018 at 01:49:12PM +0200, Gábor Boskovits wrote:
> > Do you think we can do something like rebuild icedtea3 with icedtea3, and
> > then rewrite the references? I guess this way the dependency loop could
> be
> > broken, as we could detach the last icedtea3 build from the packages used
> > for bootstrapping.
>
> If the references are not important (that is, if they are not used), we
> could also rewrite them to something that doesn't exist. Currently the
> go-build-system does this so that Go executables don't refer to the Go
> compiler, which is very large.
>

Ok, I will have a look at his, and try to isolate these references. This
looks like a
good first step to evaluate what can be done next.

[Message part 2 (text/html, inline)]

Message #17 received at 31719 <at> debbugs.gnu.org (full text, mbox):

From: Ricardo Wurmus <rekado <at> elephly.net>
To: Gábor Boskovits <boskovits <at> gmail.com>
Cc: 31719 <at> debbugs.gnu.org
Subject: Re: icedtea-3 binaries contain references to icedtea-2
Date: Tue, 05 Jun 2018 14:36:51 +0200

Gábor Boskovits <boskovits <at> gmail.com> writes:

> Do you think we can do something like rebuild icedtea3 with icedtea3, and
> then rewrite the references? I guess this way the dependency loop could be
> broken, as we could detach the last icedtea3 build from the packages used
> for bootstrapping.
> We could even make on more rebuilding, if necessary. WDYT?

Yes, that’s an option, but I’m not sure it is the best option.  Icedtea
builds already take a very long time.  Rebuilding the JDK after the
“build” phase would almost double the build time.

I would be happier if we could prevent references to the bootstrap JDK
from ending up in the binaries.  Before we can do that we need to
understand why they are there.  Are they necessary or are they just
accidental?

--
Ricardo

Message #20 received at 31719 <at> debbugs.gnu.org (full text, mbox):

From: Gábor Boskovits <boskovits <at> gmail.com>
To: 31719 <at> debbugs.gnu.org, Ricardo Wurmus <rekado <at> elephly.net>, 
 Leo Famulari <leo <at> famulari.name>
Subject: icedtea-3 binaries contain references to icedtea-2
Date: Mon, 10 Sep 2018 23:00:52 +0200

[Message part 1 (text/plain, inline)]

It seems that this is caused by having the files in a directory that is not
stripped.
I will try to have another look at that.

[Message part 2 (text/html, inline)]

Message #23 received at 31719 <at> debbugs.gnu.org (full text, mbox):

From: Ricardo Wurmus <rekado <at> elephly.net>
To: 31719 <at> debbugs.gnu.org
Cc: Gábor Boskovits <boskovits <at> gmail.com>
Subject: icedtea-3 binaries contain references to icedtea-2
Date: Thu, 14 May 2020 20:02:42 +0200

This seems to affect the openjdk packages as well, so a user of OpenJDK
12 will have to download *all* JDKs.

Gábor, have you been able to identify locations that retain references
to the build JDK?

-- 
Ricardo

Message #26 received at 31719 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Enge <andreas <at> enge.fr>
To: 31719 <at> debbugs.gnu.org
Subject: Chains of dependencies getting longer
Date: Sat, 27 Feb 2021 12:17:44 +0100

Hello,

trying a "guix build openjdk", I see this:
2136.0 MB would be downloaded:
   /gnu/store/243algr6h60j46spn5dqhjc4mhkd0a0p-libelf-0.8.13
   /gnu/store/ymyvcc0871lwbnkv5acf5lldv0ahm5z1-openjdk-9.181-jdk
   /gnu/store/3hb80r2l48ix8pq2kvcsr1i9aj76d681-openjdk-9.181
   /gnu/store/52bp22jrvwfqa5fmlv2y5vxcir74m8jb-openjdk-10.46-jdk
   /gnu/store/bp86qd2vgaqahqam3n89xl1wz02g0srs-openjdk-10.46
   /gnu/store/a8f5ksk3gb4i1wrn7m5fm5ijc0zwwg2w-openjdk-11.28-jdk
   /gnu/store/9ni61z4nhvkrxbzj6gwip8zrb7zbkx6l-openjdk-11.28
   /gnu/store/837pl2vmq9haiw3kmsnh72fkwvcljipk-openjdk-12.33-jdk
   /gnu/store/ll5dgf42x9smclrc1mx4gyk3dl683m77-openjdk-13.0-jdk
   /gnu/store/bjvy5ab6f0gff52m808vzcn3rw55yxjn-openjdk-14.0-doc
   /gnu/store/r8kxk031fighc4dxr57nl0wirzhk8v08-openjdk-14.0-jdk
   /gnu/store/25kx1vna763xh4mpszp4p4sy4j6iifs3-openjdk-14.0

So the problem is getting worse and worse over time, which makes a solution
more important.

Andreas

Message #29 received at 31719 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Andreas Enge <andreas <at> enge.fr>
Cc: 31719 <at> debbugs.gnu.org
Subject: Re: bug#31719: Chains of dependencies getting longer
Date: Mon, 01 Mar 2021 11:04:02 +0100

Hi,

Andreas Enge <andreas <at> enge.fr> skribis:

> trying a "guix build openjdk", I see this:
> 2136.0 MB would be downloaded:
>    /gnu/store/243algr6h60j46spn5dqhjc4mhkd0a0p-libelf-0.8.13
>    /gnu/store/ymyvcc0871lwbnkv5acf5lldv0ahm5z1-openjdk-9.181-jdk
>    /gnu/store/3hb80r2l48ix8pq2kvcsr1i9aj76d681-openjdk-9.181
>    /gnu/store/52bp22jrvwfqa5fmlv2y5vxcir74m8jb-openjdk-10.46-jdk
>    /gnu/store/bp86qd2vgaqahqam3n89xl1wz02g0srs-openjdk-10.46
>    /gnu/store/a8f5ksk3gb4i1wrn7m5fm5ijc0zwwg2w-openjdk-11.28-jdk
>    /gnu/store/9ni61z4nhvkrxbzj6gwip8zrb7zbkx6l-openjdk-11.28
>    /gnu/store/837pl2vmq9haiw3kmsnh72fkwvcljipk-openjdk-12.33-jdk
>    /gnu/store/ll5dgf42x9smclrc1mx4gyk3dl683m77-openjdk-13.0-jdk
>    /gnu/store/bjvy5ab6f0gff52m808vzcn3rw55yxjn-openjdk-14.0-doc
>    /gnu/store/r8kxk031fighc4dxr57nl0wirzhk8v08-openjdk-14.0-jdk
>    /gnu/store/25kx1vna763xh4mpszp4p4sy4j6iifs3-openjdk-14.0

Weird.  I see this:

--8<---------------cut here---------------start------------->8---
$ guix build openjdk -n
369.8 MB would be downloaded:
   /gnu/store/676kcvxnnjfcy960fy4xkr10fbn3ciay-openjdk-14.0-doc
   /gnu/store/dk8rkrfkdgsmy1a33n0sc39fhqggy1nh-openjdk-14.0-jdk
   /gnu/store/ylakfs5fwq5sqsk5zbkwwf2d1lp0jfxs-openjdk-14.0
$ guix build openjdk <at> 13 -n
358.0 MB would be downloaded:
   /gnu/store/hipy0g8zpnicqcwhx5ygx9fmmkbgbw6w-openjdk-13.0-doc
   /gnu/store/ff9nn7ipbpnrrvsg0djdrzm0a8v1jx5j-openjdk-13.0-jdk
   /gnu/store/nb6p8aqjjw923vwd7safbh0w5q3mr9fq-openjdk-13.0
$ guix size openjdk <at> 14 | grep openjdk
/gnu/store/ylakfs5fwq5sqsk5zbkwwf2d1lp0jfxs-openjdk-14.0           389.3   295.8  76.0%
$ guix describe
Generacio 175   Feb 04 2021 22:52:40    (nuna)
  guix 5ae09d7
    repository URL: https://git.savannah.gnu.org/git/guix.git
    branch: master
    commit: 5ae09d7979a0696d862b9555314eab199f7ce576
--8<---------------cut here---------------end--------------->8---

What this shows is that the compiled openjdk 14 does not depend on
previous versions; likewise for version 13.

So the long dependency chain is a built-time-only dependency chain.
It’s a problem when building from source but not when using substitutes.

Ludo’.

Message #32 received at 31719 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Enge <andreas <at> enge.fr>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 31719 <at> debbugs.gnu.org
Subject: Re: bug#31719: Chains of dependencies getting longer
Date: Mon, 1 Mar 2021 13:00:56 +0100

Hello,

Am Mon, Mar 01, 2021 at 11:04:02AM +0100 schrieb Ludovic Courtès:
> Andreas Enge <andreas <at> enge.fr> skribis:
> > trying a "guix build openjdk", I see this:
> > 2136.0 MB would be downloaded
...
> >    /gnu/store/ymyvcc0871lwbnkv5acf5lldv0ahm5z1-openjdk-9.181-jdk
> >    /gnu/store/3hb80r2l48ix8pq2kvcsr1i9aj76d681-openjdk-9.181
...
> >    /gnu/store/bjvy5ab6f0gff52m808vzcn3rw55yxjn-openjdk-14.0-doc
> >    /gnu/store/r8kxk031fighc4dxr57nl0wirzhk8v08-openjdk-14.0-jdk
> >    /gnu/store/25kx1vna763xh4mpszp4p4sy4j6iifs3-openjdk-14.0
> 
> Weird.  I see this:
> 
> --8<---------------cut here---------------start------------->8---
> $ guix build openjdk -n
> 369.8 MB would be downloaded:
>    /gnu/store/676kcvxnnjfcy960fy4xkr10fbn3ciay-openjdk-14.0-doc
>    /gnu/store/dk8rkrfkdgsmy1a33n0sc39fhqggy1nh-openjdk-14.0-jdk
>    /gnu/store/ylakfs5fwq5sqsk5zbkwwf2d1lp0jfxs-openjdk-14.0

It is strange we do not see the same thing! I just tried it again, and am
getting the same result as above. Notice that openjdk <at> 14 is available as a
substitute, but still all previous versions are downloaded, while nothing
is built. The same with the most recent master commit (7ca43b0a1e).

Andreas

Message #35 received at 31719 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Andreas Enge <andreas <at> enge.fr>
Cc: Björn Höfling <bjoern.hoefling <at> bjoernhoefling.de>,
 31719 <at> debbugs.gnu.org
Subject: Re: bug#31719: Chains of dependencies getting longer
Date: Mon, 01 Mar 2021 22:33:55 +0100

Hi!

Andreas Enge <andreas <at> enge.fr> skribis:

> Am Mon, Mar 01, 2021 at 11:04:02AM +0100 schrieb Ludovic Courtès:
>> Andreas Enge <andreas <at> enge.fr> skribis:
>> > trying a "guix build openjdk", I see this:
>> > 2136.0 MB would be downloaded
> ...
>> >    /gnu/store/ymyvcc0871lwbnkv5acf5lldv0ahm5z1-openjdk-9.181-jdk
>> >    /gnu/store/3hb80r2l48ix8pq2kvcsr1i9aj76d681-openjdk-9.181
> ...
>> >    /gnu/store/bjvy5ab6f0gff52m808vzcn3rw55yxjn-openjdk-14.0-doc
>> >    /gnu/store/r8kxk031fighc4dxr57nl0wirzhk8v08-openjdk-14.0-jdk
>> >    /gnu/store/25kx1vna763xh4mpszp4p4sy4j6iifs3-openjdk-14.0
>> 
>> Weird.  I see this:
>> 
>> --8<---------------cut here---------------start------------->8---
>> $ guix build openjdk -n
>> 369.8 MB would be downloaded:
>>    /gnu/store/676kcvxnnjfcy960fy4xkr10fbn3ciay-openjdk-14.0-doc
>>    /gnu/store/dk8rkrfkdgsmy1a33n0sc39fhqggy1nh-openjdk-14.0-jdk
>>    /gnu/store/ylakfs5fwq5sqsk5zbkwwf2d1lp0jfxs-openjdk-14.0
>
> It is strange we do not see the same thing! I just tried it again, and am
> getting the same result as above. Notice that openjdk <at> 14 is available as a
> substitute, but still all previous versions are downloaded, while nothing
> is built.

Wait, with a newer commit, I get the whole chain as well:

--8<---------------cut here---------------start------------->8---
$ guix build openjdk -n
2,135.0 MB would be downloaded:
   /gnu/store/ymyvcc0871lwbnkv5acf5lldv0ahm5z1-openjdk-9.181-jdk
   /gnu/store/3hb80r2l48ix8pq2kvcsr1i9aj76d681-openjdk-9.181
   /gnu/store/52bp22jrvwfqa5fmlv2y5vxcir74m8jb-openjdk-10.46-jdk
   /gnu/store/bp86qd2vgaqahqam3n89xl1wz02g0srs-openjdk-10.46
   /gnu/store/a8f5ksk3gb4i1wrn7m5fm5ijc0zwwg2w-openjdk-11.28-jdk
   /gnu/store/9ni61z4nhvkrxbzj6gwip8zrb7zbkx6l-openjdk-11.28
   /gnu/store/837pl2vmq9haiw3kmsnh72fkwvcljipk-openjdk-12.33-jdk
   /gnu/store/ll5dgf42x9smclrc1mx4gyk3dl683m77-openjdk-13.0-jdk
   /gnu/store/bjvy5ab6f0gff52m808vzcn3rw55yxjn-openjdk-14.0-doc
   /gnu/store/r8kxk031fighc4dxr57nl0wirzhk8v08-openjdk-14.0-jdk
   /gnu/store/25kx1vna763xh4mpszp4p4sy4j6iifs3-openjdk-14.0
$ guix describe
Generacio 176   Mar 01 2021 11:39:49    (nuna)
  guix 7ca43b0
    repository URL: https://git.savannah.gnu.org/git/guix.git
    branch: master
    commit: 7ca43b0a1e2215abe0df0708f31decace8e68911
--8<---------------cut here---------------end--------------->8---

But again, from that Feb. 4th commit, everything looks fine:

--8<---------------cut here---------------start------------->8---
$ guix time-machine --commit=5ae09d7979a0696d862b9555314eab199f7ce576 -- build openjdk -n
369.8 MB would be downloaded:
   /gnu/store/676kcvxnnjfcy960fy4xkr10fbn3ciay-openjdk-14.0-doc
   /gnu/store/dk8rkrfkdgsmy1a33n0sc39fhqggy1nh-openjdk-14.0-jdk
   /gnu/store/ylakfs5fwq5sqsk5zbkwwf2d1lp0jfxs-openjdk-14.0
--8<---------------cut here---------------end--------------->8---

It must be a recent regression.  I think this is an unintended
side-effect of 44425e1f2a96aee39a21dff634abb9b77b44261e and
84805ef205b7fa12bfaa7b23da06993cab64c40b (see
<https://issues.guix.gnu.org/41177>):

--8<---------------cut here---------------start------------->8---
$ guix time-machine --commit=44425e1f2a96aee39a21dff634abb9b77b44261e -- build openjdk -n
2,135.9 MB would be downloaded:
   /gnu/store/wd9jyzy7i2dxbxahc0pa8mq22any08zx-openjdk-9.181-jdk
   /gnu/store/7zxhm2xmyqds8hvlhifi9d16p9ln9k4b-openjdk-9.181
   /gnu/store/j5sh6c7qhm6vis6fvjyxs2jch9dhz085-openjdk-10.46-jdk
   /gnu/store/7w3vd4y0jac7d74gw62qmfv7br11k64r-openjdk-10.46
   /gnu/store/0sb9z8ds8ly7pwmviaqixq9wq1rp13g5-openjdk-11.28-jdk
   /gnu/store/4nf35d4v363bxrfmp85sk671swzmbj5k-openjdk-11.28
   /gnu/store/iwi28gcfxrsq541367m4hnp44jnnrkdq-openjdk-12.33-jdk
   /gnu/store/2w5gg7v9sy54s1jfnymgvg6qwqnkzr8h-openjdk-13.0-jdk
   /gnu/store/sgqw343zlqqn4vpdq4v9m4jy6ar9i99f-openjdk-14.0-doc
   /gnu/store/8l4pb8rh9zbqjn14ipbps7nn8wd0jg4w-openjdk-14.0-jdk
   /gnu/store/1d5bzdzrir8k13bkclx3a9cq59gw5j4h-openjdk-14.0
--8<---------------cut here---------------end--------------->8---

Björn, WDYT?  I suppose we need to filter out some library names in
‘patch-jni-libs’?

While at it, we should add a #:disallowed-references keyword to prevent
that from popping up in the future.

Thanks,
Ludo’.

Message #38 received at 31719 <at> debbugs.gnu.org (full text, mbox):

From: Björn Höfling <bjoern.hoefling <at> bjoernhoefling.de>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Andreas Enge <andreas <at> enge.fr>, 31719 <at> debbugs.gnu.org
Subject: Re: bug#31719: Chains of dependencies getting longer
Date: Mon, 1 Mar 2021 23:15:34 +0100

[Message part 1 (text/plain, inline)]

On Mon, 01 Mar 2021 22:33:55 +0100
Ludovic Courtès <ludo <at> gnu.org> wrote:

> It must be a recent regression.  I think this is an unintended
> side-effect of 44425e1f2a96aee39a21dff634abb9b77b44261e and
> 84805ef205b7fa12bfaa7b23da06993cab64c40b (see
> <https://issues.guix.gnu.org/41177>):
> 
> --8<---------------cut here---------------start------------->8---
> $ guix time-machine --commit=44425e1f2a96aee39a21dff634abb9b77b44261e
> -- build openjdk -n 2,135.9 MB would be downloaded:
>    /gnu/store/wd9jyzy7i2dxbxahc0pa8mq22any08zx-openjdk-9.181-jdk
>    /gnu/store/7zxhm2xmyqds8hvlhifi9d16p9ln9k4b-openjdk-9.181
>    /gnu/store/j5sh6c7qhm6vis6fvjyxs2jch9dhz085-openjdk-10.46-jdk
>    /gnu/store/7w3vd4y0jac7d74gw62qmfv7br11k64r-openjdk-10.46
>    /gnu/store/0sb9z8ds8ly7pwmviaqixq9wq1rp13g5-openjdk-11.28-jdk
>    /gnu/store/4nf35d4v363bxrfmp85sk671swzmbj5k-openjdk-11.28
>    /gnu/store/iwi28gcfxrsq541367m4hnp44jnnrkdq-openjdk-12.33-jdk
>    /gnu/store/2w5gg7v9sy54s1jfnymgvg6qwqnkzr8h-openjdk-13.0-jdk
>    /gnu/store/sgqw343zlqqn4vpdq4v9m4jy6ar9i99f-openjdk-14.0-doc
>    /gnu/store/8l4pb8rh9zbqjn14ipbps7nn8wd0jg4w-openjdk-14.0-jdk
>    /gnu/store/1d5bzdzrir8k13bkclx3a9cq59gw5j4h-openjdk-14.0
> --8<---------------cut here---------------end--------------->8---
> 
> Björn, WDYT?  I suppose we need to filter out some library names in
> ‘patch-jni-libs’?

Ouch. While installing a JDK last week, I had the same feeling that the
dependency chain down was a bit too long, but then forgot to look at it
again. 

When I read today Andreas' email, it popped up again ... I will have a
look at it.

> While at it, we should add a #:disallowed-references keyword to
> prevent that from popping up in the future.

I wasn't aware of this and will also have a look.

Björn

[Message part 2 (application/pgp-signature, inline)]

Message #41 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ricardo Wurmus <rekado <at> elephly.net>
To: Björn Höfling <bjoern.hoefling <at> bjoernhoefling.de>
Cc: Ludovic Courtès <ludo <at> gnu.org>, bug-guix <at> gnu.org,
 31719 <at> debbugs.gnu.org
Subject: Re: bug#31719: Chains of dependencies getting longer
Date: Wed, 14 Apr 2021 09:36:23 +0200

Hi Björn,

> On Mon, 01 Mar 2021 22:33:55 +0100
> Ludovic Courtès <ludo <at> gnu.org> wrote:
>
>> It must be a recent regression.  I think this is an unintended
>> side-effect of 44425e1f2a96aee39a21dff634abb9b77b44261e and
>> 84805ef205b7fa12bfaa7b23da06993cab64c40b (see
>> <https://issues.guix.gnu.org/41177>):
>> 
>> --8<---------------cut 
>> here---------------start------------->8---
>> $ guix time-machine 
>> --commit=44425e1f2a96aee39a21dff634abb9b77b44261e
>> -- build openjdk -n 2,135.9 MB would be downloaded:
>>    /gnu/store/wd9jyzy7i2dxbxahc0pa8mq22any08zx-openjdk-9.181-jdk
>>    /gnu/store/7zxhm2xmyqds8hvlhifi9d16p9ln9k4b-openjdk-9.181
>>    /gnu/store/j5sh6c7qhm6vis6fvjyxs2jch9dhz085-openjdk-10.46-jdk
>>    /gnu/store/7w3vd4y0jac7d74gw62qmfv7br11k64r-openjdk-10.46
>>    /gnu/store/0sb9z8ds8ly7pwmviaqixq9wq1rp13g5-openjdk-11.28-jdk
>>    /gnu/store/4nf35d4v363bxrfmp85sk671swzmbj5k-openjdk-11.28
>>    /gnu/store/iwi28gcfxrsq541367m4hnp44jnnrkdq-openjdk-12.33-jdk
>>    /gnu/store/2w5gg7v9sy54s1jfnymgvg6qwqnkzr8h-openjdk-13.0-jdk
>>    /gnu/store/sgqw343zlqqn4vpdq4v9m4jy6ar9i99f-openjdk-14.0-doc
>>    /gnu/store/8l4pb8rh9zbqjn14ipbps7nn8wd0jg4w-openjdk-14.0-jdk
>>    /gnu/store/1d5bzdzrir8k13bkclx3a9cq59gw5j4h-openjdk-14.0
>> --8<---------------cut 
>> here---------------end--------------->8---
>> 
>> Björn, WDYT?  I suppose we need to filter out some library 
>> names in
>> ‘patch-jni-libs’?
>
> Ouch. While installing a JDK last week, I had the same feeling 
> that the
> dependency chain down was a bit too long, but then forgot to 
> look at it
> again. 
>
> When I read today Andreas' email, it popped up again ... I will 
> have a
> look at it.
>
>> While at it, we should add a #:disallowed-references keyword to
>> prevent that from popping up in the future.
>
> I wasn't aware of this and will also have a look.

Have you been able to identify what’s wrong with the JDK chain? 
I’d like us to drop these extra references soon so that they don’t 
appear in the upcoming release.

-- 
Ricardo

Message #47 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Björn Höfling <bjoern.hoefling <at> bjoernhoefling.de>
To: Ricardo Wurmus <rekado <at> elephly.net>
Cc: Ludovic Courtès <ludo <at> gnu.org>, bug-guix <at> gnu.org,
 31719 <at> debbugs.gnu.org
Subject: Re: bug#31719: Chains of dependencies getting longer
Date: Fri, 16 Apr 2021 21:22:33 +0200

[Message part 1 (text/plain, inline)]

Hi Ricardo,

On Wed, 14 Apr 2021 09:36:23 +0200
Ricardo Wurmus <rekado <at> elephly.net> wrote:

> Have you been able to identify what’s wrong with the JDK chain? 
> I’d like us to drop these extra references soon so that they don’t 
> appear in the upcoming release.

Sorry, I currently don't find the time to look at the problem. Would
someone else give it a try?

Björn

[Message part 2 (application/pgp-signature, inline)]

Message #53 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Carlo Zancanaro <carlo <at> zancanaro.id.au>
To: Björn Höfling <bjoern.hoefling <at> bjoernhoefling.de>
Cc: Ricardo Wurmus <rekado <at> elephly.net>, 31719 <at> debbugs.gnu.org,
 bug-guix <at> gnu.org
Subject: Re: bug#31719: Chains of dependencies getting longer
Date: Sat, 17 Apr 2021 08:26:05 +1000

On Sat, Apr 17 2021, Björn Höfling wrote:
> Sorry, I currently don't find the time to look at the problem. 
> Would someone else give it a try?

I just had a quick look, and I think we can resolve it by changing 
the patch-jni-libs to not rewrite references to "mlib_image" and 
"splashscreen". It looks like they're provided as part of the JVM 
built itself, but our build phase hard codes a reference to the 
previous JVM's build result instead of using the current JVM's 
build result. They're the only dependencies I've found so far, but 
I've only looked at openjdk10 and openjdk14.

I'll put together a patch later today, if I haven't been beaten to 
it by then.

Carlo

Message #59 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Carlo Zancanaro <carlo <at> zancanaro.id.au>
Cc: Björn Höfling <bjoern.hoefling <at> bjoernhoefling.de>,
 rekado <at> elephly.net, 31719 <at> debbugs.gnu.org, bug-guix <at> gnu.org
Subject: Re: bug#31719: Chains of dependencies getting longer
Date: Sat, 17 Apr 2021 17:11:20 +1000

[Message part 1 (text/plain, inline)]

Here's a patch that should clean up these runtime dependencies. 
It's a bit specific to this particular case, but I think that 
might be fine for now.

I think it would make more sense for native inputs to not have 
their paths included in LIBRARY_PATH. Does it even make sense for 
them to be there? I thought LIBRARY_PATH was for compilers to find 
dependencies when compiling so they can link their output binaries 
against them. Having native inputs show up there seems wrong.

I'm in the process of rebuilding Java from icedtea-8 upwards to 
check, but I have already tested that modifying openjdk 9 and 10 
leads to "guix gc --references" show that openjdk 10 does not 
depend on openjdk 9. I have also tested that I can run some 
complex Java programs on my machine using the openjdk 10 built 
using this patch.

Carlo

[0001-gnu-Clean-up-runtime-dependencies-between-Java-versi.patch (text/x-diff, attachment)]

Message #65 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Carlo Zancanaro <carlo <at> zancanaro.id.au>
Cc: rekado <at> elephly.net, 31719 <at> debbugs.gnu.org, bug-guix <at> gnu.org
Subject: Re: bug#31719: Chains of dependencies getting longer
Date: Sat, 17 Apr 2021 19:38:11 +1000

On Sat, Apr 17 2021, Carlo Zancanaro wrote:
> I'm in the process of rebuilding Java from icedtea-8 upwards to 
> check,

I've now built and checked the JRE/JDKs from 10 to 14, and none of 
them retain a reference to any other JRE/JDK according to "guix gc 
--references".

Carlo

Message #71 received at 31719 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Enge <andreas <at> enge.fr>
To: Carlo Zancanaro <carlo <at> zancanaro.id.au>
Cc: rekado <at> elephly.net, 31719 <at> debbugs.gnu.org
Subject: Re: bug#31719: Chains of dependencies getting longer
Date: Mon, 19 Apr 2021 18:22:24 +0200

Hello Carlo,

Am Sat, Apr 17, 2021 at 07:38:11PM +1000 schrieb Carlo Zancanaro:
> On Sat, Apr 17 2021, Carlo Zancanaro wrote:
> > I'm in the process of rebuilding Java from icedtea-8 upwards to check,
> 
> I've now built and checked the JRE/JDKs from 10 to 14, and none of them
> retain a reference to any other JRE/JDK according to "guix gc --references".

thanks a lot for your patch! I am not very knowledgeable on this matter, but
after updating my system am right now once again bitten by the problem:
The following derivations would be built:
   /gnu/store/0bxpgqps79007jky37dwzgm08wlsgj02-openjdk-10.46.drv
   /gnu/store/2xwsm2plrbk7l4510hrqp9y7b5vv9v5j-module-import-compiled.drv
1683.5 MB would be downloaded:
   /gnu/store/0gzv5208m2prqbnsqdffcnz7mwfqa684-openjdk-10.46.tar.xz
   /gnu/store/5g6ng9a2ifgrv57mzdaysf2lq9rkixxk-make-4.2.1
   /gnu/store/243algr6h60j46spn5dqhjc4mhkd0a0p-libelf-0.8.13
   /gnu/store/crc4cgsdls10isy8prjdddvsp21wafx6-openjdk-9.181-jdk
   /gnu/store/6amc3php7qyqxagak5xnmv2k81wm7w26-openjdk-9.181
   /gnu/store/bms6jg5qwn927qmbh9xi30ifhsxxdazq-openjdk-11.28-jdk
   /gnu/store/lylv6ng5gwqpi930c6y7sglh2k8byjn6-openjdk-11.28
   /gnu/store/z27mhqpylrbmwkcgrb100p6jbflxhq5h-openjdk-12.33-jdk
   /gnu/store/xz500ry11dihwn9kij1kmzkci1lmnqjf-openjdk-13.0-jdk
   /gnu/store/a1s66hlj10nkkcxlk6s2dzq4iip4mh4k-openjdk-14.0-doc
   /gnu/store/cyssia0a5lh8mb5czd7155y7sl31aryl-openjdk-14.0-jdk
   /gnu/store/630lvja8vak1bkf9vsbbh29cp79j4pwp-openjdk-14.0
guix build: warning: at least 5922.4 MB needed but only 3539.4 MB available in /gnu/store

So I would have to download lots of openjdk and even build one in the middle
of the chain myself.

So unless someone else objects, I intend to apply the patch, build the
latest openjdk, and push it if everything goes well.

Notice that the original problem reported by Ricardo seems to have been
fixed independently - "guix build icedtea" only downloads icedtea <at> 3, and
not @2 on current master.

Thanks,

Andreas

Message #78 received at 31719 <at> debbugs.gnu.org (full text, mbox):

From: Ricardo Wurmus <rekado <at> elephly.net>
To: Andreas Enge <andreas <at> enge.fr>
Cc: Carlo Zancanaro <carlo <at> zancanaro.id.au>, 31719 <at> debbugs.gnu.org
Subject: Re: bug#31719: Chains of dependencies getting longer
Date: Mon, 19 Apr 2021 20:18:03 +0200

Andreas Enge <andreas <at> enge.fr> writes:

> So unless someone else objects, I intend to apply the patch, 
> build the
> latest openjdk, and push it if everything goes well.

Sounds good.

I just looked over the patch, and while I’m not sure it’s the best 
way to do things (matching “openjdk” or “icedtea” in the package 
name seems a little error prone in the presence of packages whose 
names might include these strings), but I think it’s a definite 
improvement.

Thank you Carlo!

-- 
Ricardo

Message #81 received at 31719 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Enge <andreas <at> enge.fr>
To: Ricardo Wurmus <rekado <at> elephly.net>
Cc: Carlo Zancanaro <carlo <at> zancanaro.id.au>, 31719 <at> debbugs.gnu.org
Subject: Re: bug#31719: Chains of dependencies getting longer
Date: Tue, 20 Apr 2021 10:34:17 +0200

Hello,

Am Mon, Apr 19, 2021 at 08:18:03PM +0200 schrieb Ricardo Wurmus:
> I just looked over the patch, and while I’m not sure it’s the best way to do
> things (matching “openjdk” or “icedtea” in the package name seems a little
> error prone in the presence of packages whose names might include these
> strings), but I think it’s a definite improvement.

I just pushed the patch to master. Thanks a lot, Carlo! It has definitely
solved my problem: I can now compile an Android project after downloading
a single openjdk package.

It would be nice if someone else could close the bug if you feel the problem
is solved, or otherwise leave it open to discuss further possible improvements.

Andreas

Message #84 received at 31719 <at> debbugs.gnu.org (full text, mbox):

From: Carlo Zancanaro <carlo <at> zancanaro.id.au>
To: Ricardo Wurmus <rekado <at> elephly.net>
Cc: 31719 <at> debbugs.gnu.org
Subject: Re: bug#31719: Chains of dependencies getting longer
Date: Tue, 20 Apr 2021 19:01:36 +1000

On 20 April 2021 4:18:03 am AEST, Ricardo Wurmus <rekado <at> elephly.net> wrote:
>I just looked over the patch, and while I’m not sure it’s the best way to do things (matching “openjdk” or “icedtea” in the package name seems a little error prone in the presence of packages whose names might include these strings), but I think it’s a definite improvement.

Yeah, I'm agreed on that. I tried for a while to find a way to exclude native inputs, but as far as I could tell they all ended up as inputs within the build phases. I'd be happy to change it if somebody can give me a hint about how to pick out the native inputs.

Carlo

Message #87 received at 31719 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Andreas Enge <andreas <at> enge.fr>
Cc: Ricardo Wurmus <rekado <at> elephly.net>,
 Carlo Zancanaro <carlo <at> zancanaro.id.au>, 31719 <at> debbugs.gnu.org
Subject: Re: bug#31719: icedtea-3 binaries contain references to icedtea-2
Date: Tue, 20 Apr 2021 11:24:59 +0200

[Message part 1 (text/plain, inline)]

Hi,

Andreas Enge <andreas <at> enge.fr> skribis:

> Am Mon, Apr 19, 2021 at 08:18:03PM +0200 schrieb Ricardo Wurmus:
>> I just looked over the patch, and while I’m not sure it’s the best way to do
>> things (matching “openjdk” or “icedtea” in the package name seems a little
>> error prone in the presence of packages whose names might include these
>> strings), but I think it’s a definite improvement.
>
> I just pushed the patch to master. Thanks a lot, Carlo! It has definitely
> solved my problem: I can now compile an Android project after downloading
> a single openjdk package.
>
> It would be nice if someone else could close the bug if you feel the problem
> is solved, or otherwise leave it open to discuss further possible improvements.

I think we can close it.  I have the attached improvements that I can
commit to ‘staging’ (or ‘core-updates’?) to avoid another rebuild.

Thanks!

Ludo’.

[0001-java-fixlet.patch (text/x-patch, inline)]

From 253d0485a9307c4e08afc058d7dafcd56025f9a6 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo <at> gnu.org>
Date: Tue, 20 Apr 2021 00:26:01 +0200
Subject: [PATCH] java fixlet

---
 gnu/packages/java.scm | 35 +++++++++++++++++++++++++++--------
 1 file changed, 27 insertions(+), 8 deletions(-)

diff --git a/gnu/packages/java.scm b/gnu/packages/java.scm
index 3c4013ab6f..b780f7a85f 100644
--- a/gnu/packages/java.scm
+++ b/gnu/packages/java.scm
@@ -1749,6 +1749,9 @@ IcedTea build harness.")
          ((guix build ant-build-system)
           (guix build syscalls)
           ,@%gnu-build-system-modules)
+
+         #:disallowed-references ((,icedtea-7 "jdk"))
+
          ,@(substitute-keyword-arguments (package-arguments icedtea-7)
              ((#:modules modules)
               `((guix build utils)
@@ -1792,10 +1795,13 @@ new Date();"))
                  (add-after 'unpack 'patch-jni-libs
                    ;; Hardcode dynamically loaded libraries.
                    (lambda _
-                     (use-modules (srfi srfi-1))
+                     (define remove
+                       (@ (srfi srfi-1) remove))
+
                      (define (icedtea-or-openjdk? path)
                        (or (string-contains path "openjdk")
                            (string-contains path "icedtea")))
+
                      (let* ((library-path (remove icedtea-or-openjdk?
                                                   (search-path-as-string->list
                                                    (getenv "LIBRARY_PATH"))))
@@ -1898,6 +1904,9 @@ new Date();"))
        #:imported-modules
        ((guix build syscalls)
         ,@%gnu-build-system-modules)
+
+       #:disallowed-references (,icedtea-8 (,icedtea-8 "jdk"))
+
        #:phases
        (modify-phases %standard-phases
          (add-after 'patch-source-shebangs 'fix-java-shebangs
@@ -1936,18 +1945,20 @@ new Date();"))
          (add-after 'unpack 'patch-jni-libs
            ;; Hardcode dynamically loaded libraries.
            (lambda _
-             (use-modules (srfi srfi-1))
+             (define remove
+               (@ (srfi srfi-1) remove))
+
              (define (icedtea-or-openjdk? path)
                (or (string-contains path "openjdk")
                    (string-contains path "icedtea")))
+
              (let* ((library-path (remove icedtea-or-openjdk?
                                           (search-path-as-string->list
                                            (getenv "LIBRARY_PATH"))))
                     (find-library (lambda (name)
-                                    (or (search-path
-                                         library-path
-                                         (string-append "lib" name ".so"))
-                                        (string-append "lib" name ".so")))))
+                                    (search-path
+                                     library-path
+                                     (string-append "lib" name ".so")))))
                (for-each
                 (lambda (file)
                   (catch 'decoding-error
@@ -2090,7 +2101,9 @@ new Date();"))
                        "--with-libjpeg=system"
                        "--with-native-debug-symbols=zipped"
                        (string-append "--prefix=" (assoc-ref outputs "out")))
-               #t))))))
+               #t))))
+       ((#:disallowed-references _ '())
+        `(,openjdk9 (,openjdk9 "jdk")))))
     (native-inputs
      `(("openjdk9" ,openjdk9)
        ("openjdk9:jdk" ,openjdk9 "jdk")
@@ -2120,6 +2133,9 @@ new Date();"))
     (arguments
      `(#:imported-modules ((guix build syscalls)
                            ,@%gnu-build-system-modules)
+
+       #:disallowed-references (,openjdk10 (,openjdk10 "jdk"))
+
        #:tests? #f; requires jtreg
        ;; TODO package jtreg
        #:configure-flags
@@ -2150,10 +2166,13 @@ new Date();"))
          (add-after 'unpack 'patch-jni-libs
            ;; Hardcode dynamically loaded libraries.
            (lambda _
-             (use-modules (srfi srfi-1))
+             (define remove
+               (@ (srfi srfi-1) remove))
+
              (define (icedtea-or-openjdk? path)
                (or (string-contains path "openjdk")
                    (string-contains path "icedtea")))
+
              (let* ((library-path (remove icedtea-or-openjdk?
                                           (search-path-as-string->list
                                            (getenv "LIBRARY_PATH"))))
-- 
2.31.1

Message #90 received at 31719 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Enge <andreas <at> enge.fr>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Ricardo Wurmus <rekado <at> elephly.net>,
 Carlo Zancanaro <carlo <at> zancanaro.id.au>, 31719 <at> debbugs.gnu.org
Subject: Re: bug#31719: icedtea-3 binaries contain references to icedtea-2
Date: Tue, 20 Apr 2021 11:36:08 +0200

Am Tue, Apr 20, 2021 at 11:24:59AM +0200 schrieb Ludovic Courtès:
> I think we can close it.  I have the attached improvements that I can
> commit to ‘staging’ (or ‘core-updates’?) to avoid another rebuild.

I think they can easily go to master; the time for rebuilds is not very
high, I just ran into trouble since I had little space left on the device
so some manual intervention was required to gc intermediate packages.

There also seem to be very few dependencies if "guix refresh -l openjdk"
is to be trusted.

Andreas

Message #93 received at 31719 <at> debbugs.gnu.org (full text, mbox):

From: Carlo Zancanaro <carlo <at> zancanaro.id.au>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Ricardo Wurmus <rekado <at> elephly.net>, Andreas Enge <andreas <at> enge.fr>,
 31719 <at> debbugs.gnu.org
Subject: Re: bug#31719: icedtea-3 binaries contain references to icedtea-2
Date: Tue, 20 Apr 2021 21:32:10 +1000

[Message part 1 (text/plain, inline)]

Hi Ludo!

On Tue, Apr 20 2021, Ludovic Courtès wrote:
>                      (find-library (lambda (name)
> -                                    (or (search-path
> -                                         library-path
> -                                         (string-append "lib" 
> name ".so"))
> -                                        (string-append "lib" 
> name ".so")))))
> +                                    (search-path
> +                                     library-path
> +                                     (string-append "lib" name 
> ".so")))))
>                 (for-each

As discussed on IRC, the "or" is actually important here to avoid 
substituting #f as the library name. I've attached a patch on top 
of yours that adds the "or" back (including the other two that I 
missed in my earlier patch), and also switches to "string-append" 
which is less sensitive to this problem.

I have built up to openjdk11 with this patch, and I see less #f's 
in the result. There are still some in the compiled libraries, but 
I haven't investigated thoroughly as to whether they're correct or 
not.

Carlo

[0001-gnu-Fix-openjdk-library-substitution-when-libraries-.patch (text/x-diff, attachment)]

Message #96 received at 31719 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Carlo Zancanaro <carlo <at> zancanaro.id.au>
Cc: Ricardo Wurmus <rekado <at> elephly.net>, Andreas Enge <andreas <at> enge.fr>,
 31719 <at> debbugs.gnu.org
Subject: Re: bug#31719: icedtea-3 binaries contain references to icedtea-2
Date: Tue, 20 Apr 2021 18:38:56 +0200

Hi Carlo,

Carlo Zancanaro <carlo <at> zancanaro.id.au> skribis:

> From 60101b27543b7cc41a052d5bec95234ea4977d35 Mon Sep 17 00:00:00 2001
> From: Carlo Zancanaro <carlo <at> zancanaro.id.au>
> Date: Tue, 20 Apr 2021 21:22:20 +1000
> Subject: [PATCH] gnu: Fix openjdk library substitution when libraries aren't
>  found
>
> * gnu/packages/java.scm (icedtea-8, openjdk9, openjdk11): Fix JNI library
> substitution to not substitute #f if the library can't be found.

Thanks for the update patch!  Please mention ‘find-library’ in the
commit log, for clarity.

I think you can go and push it to master given that OpenJDK is probably
broken due to those #f right now.  However, please push an additional
commit at the same time that adds those #:disallowed-references I
proposed.  (Though you’ll have to rebuild openjdk <at> 11 to be sure.)

Sounds good?

Ludo’.

Message #99 received at 31719 <at> debbugs.gnu.org (full text, mbox):

From: Carlo Zancanaro <carlo <at> zancanaro.id.au>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 31719 <at> debbugs.gnu.org
Subject: Re: bug#31719: icedtea-3 binaries contain references to icedtea-2
Date: Wed, 21 Apr 2021 07:00:11 +1000

On 21 April 2021 2:38:56 am AEST, "Ludovic Courtès" <ludo <at> gnu.org> wrote:
>Sounds good?

Sounds good, except I don't have commit privileges. 😊 Can someone else push it for me?

Message #104 received at 31719-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Carlo Zancanaro <carlo <at> zancanaro.id.au>
Cc: 31719-done <at> debbugs.gnu.org
Subject: Re: bug#31719: icedtea-3 binaries contain references to icedtea-2
Date: Wed, 21 Apr 2021 14:40:40 +0200

Hi,

Carlo Zancanaro <carlo <at> zancanaro.id.au> skribis:

> On 21 April 2021 2:38:56 am AEST, "Ludovic Courtès" <ludo <at> gnu.org> wrote:
>>Sounds good?
>
> Sounds good, except I don't have commit privileges. 😊 Can someone else push it for me?

Oops!  I adjusted the patch so it would apply to ‘master’, and followed
up with two commits:

  f8acd1aeef gnu: openjdk: Disallow references to the JDK used for build.
  e511a1d327 gnu: openjdk: Avoid non-top-level 'use-modules'.
  698c4365ba gnu: openjdk: Fix library substitution when libraries aren't found.

I rebuilt everything up to openjdk <at> 11.

Thanks!

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.