GNU bug report logs - #14851
linux-libre-3.3.8-gnu disappeared

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 14851 in the body.
You can then email your comments to 14851 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Andreas Enge <andreas <at> enge.fr>
To: bug-guix <at> gnu.org
Subject: linux-libre-3.3.8-gnu disappeared
Date: Fri, 12 Jul 2013 13:02:05 +0200

The build of linux-libre-headers-3.3.8 currently fails because the tarball 
cannot be downloaded any more from
   http://linux-libre.fsfla.org/pub/linux-libre/releases/3.3.8-gnu/
Instead, there is a new version in
   http://linux-libre.fsfla.org/pub/linux-libre/releases/3.3.8-gnu1/

Should we switch to this one? Or update to the newest version? In all 
cases, I am afraid a complete rebuild will be triggered, so we might as 
well advance in the version numbers.

Andreas

Message #8 received at 14851 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Andreas Enge <andreas <at> enge.fr>, Alexandre Oliva <lxoliva <at> fsfla.org>
Cc: 14851 <at> debbugs.gnu.org
Subject: Re: bug#14851: linux-libre-3.3.8-gnu disappeared
Date: Sat, 13 Jul 2013 00:30:02 +0200

Hello,

(Cc: Alexandre Oliva of Linux-Libre.)

Andreas Enge <andreas <at> enge.fr> skribis:

> The build of linux-libre-headers-3.3.8 currently fails because the tarball 
> cannot be downloaded any more from
>    http://linux-libre.fsfla.org/pub/linux-libre/releases/3.3.8-gnu/
> Instead, there is a new version in
>    http://linux-libre.fsfla.org/pub/linux-libre/releases/3.3.8-gnu1/

I just checked and the latter has a different SHA256
(1860as3dwh7f3im1sq3x06awmil9vppl6igg2gnva5w9mbkw2fc8).

> Should we switch to this one? Or update to the newest version? In all 
> cases, I am afraid a complete rebuild will be triggered, so we might as 
> well advance in the version numbers.

Indeed.

Since we’re about to release a new version of Guix, I’d rather keep
using 3.3.8.

Alexandre: could you reinstate the original
http://linux-libre.fsfla.org/pub/linux-libre/releases/3.3.8-gnu/linux-libre-3.3.8-gnu.tar.xz?

It would be ideal if the tarballs were on ftp.gnu.org.  I could do it if
you don’t want to bother, provided the FTP admins allow it.  WDYT?

Thanks,
Ludo’.

Message #11 received at 14851 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: "Jason Self" <jason <at> bluehome.net>
Cc: 14851 <at> debbugs.gnu.org, Andreas Enge <andreas <at> enge.fr>,
 Alexandre Oliva <lxoliva <at> fsfla.org>
Subject: Re: bug#14851: linux-libre-3.3.8-gnu disappeared
Date: Sun, 14 Jul 2013 15:55:39 +0200

"Jason Self" <jason <at> bluehome.net> skribis:

> My understanding is that the change from gnu to gnu1 is the result of
> changes that were needed to the deblobbing. Specifically to allow the
> loading of the firmware used by ath9k-htc, now that it's free [0]...
> i.e. 3.3.8-gnu and 3.3.8-gnu1 aren't the same source code.

OK, thanks for the info.

It’s hard for us to deal with disappearing software.  In this case,
we’re currently using this tarball just to install the Linux-Libre
headers to build libc, and our ‘linux-libre’ package can use a different
upstream version than ‘linux-libre-headers’ anyway.

Thanks,
Ludo’.

Message #14 received at 14851 <at> debbugs.gnu.org (full text, mbox):

From: "Jason Self" <jason <at> bluehome.net>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 14851 <at> debbugs.gnu.org, Andreas Enge <andreas <at> enge.fr>,
 Alexandre Oliva <lxoliva <at> fsfla.org>
Subject: Re: bug#14851: linux-libre-3.3.8-gnu disappeared
Date: Sat, 13 Jul 2013 21:28:07 -0700 (PDT)

[Message part 1 (text/plain, inline)]

My understanding is that the change from gnu to gnu1 is the result of
changes that were needed to the deblobbing. Specifically to allow the
loading of the firmware used by ath9k-htc, now that it's free [0]...
i.e. 3.3.8-gnu and 3.3.8-gnu1 aren't the same source code.

[0]
https://www.fsf.org/news/ryf-certification-thinkpenguin-usb-with-atheros-chip

Message #17 received at 14851 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: 14851 <at> debbugs.gnu.org
Cc: Andreas Enge <andreas <at> enge.fr>, Alexandre Oliva <lxoliva <at> fsfla.org>,
 Jason Self <jason <at> bluehome.net>
Subject: Re: bug#14851: linux-libre-3.3.8-gnu disappeared
Date: Wed, 17 Jul 2013 15:31:42 +0200

I’ve asked ftp-upload@ if they could allow me to upload linux-libre
tarballs to ftp.gnu.org.

In the meantime, I’ve uploaded this tarball to
ftp://alpha.gnu.org/gnu/guix/mirror and changed linux.scm to refer to it.

Thanks,
Ludo’.

Message #20 received at 14851 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Alexandre Oliva <lxoliva <at> fsfla.org>
Cc: 14851 <at> debbugs.gnu.org, Andreas Enge <andreas <at> enge.fr>
Subject: Re: bug#14851: linux-libre-3.3.8-gnu disappeared
Date: Fri, 19 Jul 2013 14:09:37 +0200

Alexandre Oliva <lxoliva <at> fsfla.org> skribis:

> On Jul 12, 2013, ludo <at> gnu.org (Ludovic Courtès) wrote:
>
>> Since we’re about to release a new version of Guix, I’d rather keep
>> using 3.3.8.
>
>> Alexandre: could you reinstate the original
>> http://linux-libre.fsfla.org/pub/linux-libre/releases/3.3.8-gnu/linux-libre-3.3.8-gnu.tar.xz?
>
> I suppose you don't want to prevent users of guix from using ath9k wifi
> cards, so I strongly suggest switching to 3.3.8-gnu1.

Of course not, but again, this one is used to get headers against which
to build glibc, so that’s not a problem.

> Indeed, I think you'd be better off with some LTS version of GNU
> Linux-libre, rather than the dead 3.3 branch.  But that's your call.

When we have a stand-alone, bootable distro, we’ll certainly want to
synchronize with you for the choice of the default kernel version.

>> It would be ideal if the tarballs were on ftp.gnu.org.  I could do it if
>> you don’t want to bother, provided the FTP admins allow it.  WDYT?
>
> I'd be glad with such an arrangement.

Great.

> Now, another possibility that I think would make more sense for guix is
> to have its sources consolidated in a single place, rather than
> scattered all over and at risk of having them pulled from under you.

Actually, our continuous integration server at http://hydra.gnu.org does
that transparently: it caches all the source tarballs, along with build
byproducts.

So when a tarball vanishes from its upstream site, it’s usually not a
blocking problem.  Yet, it’s preferable to have them elsewhere, because
they will eventually be garbage-collected from hydra.gnu.org.

[...]

> When we get GNU Linux-libre at ftp.gnu.org, it could then be hard links,
> so that if we remove some tarball it won't go away from your “copy”, but
> until then, you might be better off holding your own copy rather than
> assuming our primary repository has infinite space.  Unfortunately it
> doesn't, and I have to clean things up quite often.  For sources, I at
> least keep enough bits around that the tarballs can be reconstructed in
> a bit-exact fashion, but for binaries, when they're gone, they're gone
> forever.  However, considering we put out multiple GBs of builds per
> week, I don't think it's realistic to keep them all forever.  Not in our
> own server, not at ftp.gnu.org.

What do you mean by “multiple GBs of builds per week”?  Linux{,-Libre}
releases are not that frequent, are they?

The policy at ftp.gnu.org has always been to keep everything forever,
AFAIK.

If size turns out to be a problem, we could choose to keep only LTS
releases on ftp.gnu.org, for instance.  That’s something to discuss
with the GNU sysadmins.

Thanks for your feedback!

Ludo’.

Message #23 received at 14851 <at> debbugs.gnu.org (full text, mbox):

From: Alexandre Oliva <lxoliva <at> fsfla.org>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 14851 <at> debbugs.gnu.org, Andreas Enge <andreas <at> enge.fr>
Subject: Re: bug#14851: linux-libre-3.3.8-gnu disappeared
Date: Fri, 19 Jul 2013 08:08:43 -0300

On Jul 12, 2013, ludo <at> gnu.org (Ludovic Courtès) wrote:

> Since we’re about to release a new version of Guix, I’d rather keep
> using 3.3.8.

> Alexandre: could you reinstate the original
> http://linux-libre.fsfla.org/pub/linux-libre/releases/3.3.8-gnu/linux-libre-3.3.8-gnu.tar.xz?

I suppose you don't want to prevent users of guix from using ath9k wifi
cards, so I strongly suggest switching to 3.3.8-gnu1.  Indeed, I think
you'd be better off with some LTS version of GNU Linux-libre, rather
than the dead 3.3 branch.  But that's your call.

> It would be ideal if the tarballs were on ftp.gnu.org.  I could do it if
> you don’t want to bother, provided the FTP admins allow it.  WDYT?

I'd be glad with such an arrangement.  I keep on failing to figure out
how to fit the weekly publishing of multiple releases into a workflow
that includes collecting information and sending it to ftp.gnu.org
without keeping another local copy of stuff that was published before.
That's one of the hold-up factors for me.  As for the tarballs, they're
all signed, so figuring out a way to upload just the bits created since
the last push, and pushing them to live, is what's missing.

Now, another possibility that I think would make more sense for guix is
to have its sources consolidated in a single place, rather than
scattered all over and at risk of having them pulled from under you.  At
the very least, you ought to keep a copy of sources you use to build
binaries you publish, so that you can satisfy your obligation to offer
the corresponding source, be it a legal (copyleft) or moral (software in
gnu ought to be free) obligation.

When we get GNU Linux-libre at ftp.gnu.org, it could then be hard links,
so that if we remove some tarball it won't go away from your “copy”, but
until then, you might be better off holding your own copy rather than
assuming our primary repository has infinite space.  Unfortunately it
doesn't, and I have to clean things up quite often.  For sources, I at
least keep enough bits around that the tarballs can be reconstructed in
a bit-exact fashion, but for binaries, when they're gone, they're gone
forever.  However, considering we put out multiple GBs of builds per
week, I don't think it's realistic to keep them all forever.  Not in our
own server, not at ftp.gnu.org.

-- 
Alexandre Oliva, freedom fighter    http://FSFLA.org/~lxoliva/
You must be the change you wish to see in the world. -- Gandhi
Be Free! -- http://FSFLA.org/   FSF Latin America board member
Free Software Evangelist      Red Hat Brazil Compiler Engineer

Message #26 received at 14851 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Enge <andreas <at> enge.fr>
To: Alexandre Oliva <lxoliva <at> fsfla.org>
Cc: 14851 <at> debbugs.gnu.org, Ludovic Courtès <ludo <at> gnu.org>
Subject: Re: bug#14851: linux-libre-3.3.8-gnu disappeared
Date: Fri, 19 Jul 2013 18:51:47 +0200

Am Freitag, 19. Juli 2013 schrieb Alexandre Oliva:
> However, considering we put out multiple GBs of builds per
> week, I don't think it's realistic to keep them all forever.  Not in our
> own server, not at ftp.gnu.org.

As far as I know, ftp.gnu.org would only be concerned with the source, that 
is, the content of 
   http://linux-libre.fsfla.org/pub/linux-libre/releases/
and not with binary packages in upper level directories, and these source 
tarballs are usually kept indefinitely.

One also need not add the patch and diff files, so that the amount of 
storage would be quite manageable.

Andreas

Message #29 received at 14851 <at> debbugs.gnu.org (full text, mbox):

From: Alexandre Oliva <lxoliva <at> fsfla.org>
To: Andreas Enge <andreas <at> enge.fr>
Cc: 14851 <at> debbugs.gnu.org, Ludovic Courtès <ludo <at> gnu.org>
Subject: Re: bug#14851: linux-libre-3.3.8-gnu disappeared
Date: Fri, 19 Jul 2013 16:40:28 -0300

On Jul 19, 2013, Andreas Enge <andreas <at> enge.fr> wrote:

> Am Freitag, 19. Juli 2013 schrieb Alexandre Oliva:
>> However, considering we put out multiple GBs of builds per
>> week, I don't think it's realistic to keep them all forever.  Not in our
>> own server, not at ftp.gnu.org.

> As far as I know, ftp.gnu.org would only be concerned with the source,

I don't know about that.  I've seen binaries in ftp.gnu.org in the past.
This should indeed reduce significantly the space requirements to a bit
less than 1GB per week, considering 4 releases per week in 3 different
compression formats.

OTOH, if we're to be strict, we should publish only the deblob scripts,
for those are the ultimate sources produced by the GNU Linux-libre
project.  The tarballs and diffs and deltas are just the result of
running those scripts on tarballs produced by third parties.

The scripts would be trivial to retain indefinitely; even more so
because they seldom change for stable releases.

> and these source tarballs are usually kept indefinitely.

> One also need not add the patch and diff files, so that the amount of 
> storage would be quite manageable.

The diff files are probably no big deal (10% increase, not an order of
magnitude like the build tarballs and debuginfo rpms).  Excluding them
would probably make for more work in the upload script, though.

-- 
Alexandre Oliva, freedom fighter    http://FSFLA.org/~lxoliva/
You must be the change you wish to see in the world. -- Gandhi
Be Free! -- http://FSFLA.org/   FSF Latin America board member
Free Software Evangelist      Red Hat Brazil Compiler Engineer

Message #32 received at 14851 <at> debbugs.gnu.org (full text, mbox):

From: Alexandre Oliva <lxoliva <at> fsfla.org>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 14851 <at> debbugs.gnu.org, Andreas Enge <andreas <at> enge.fr>
Subject: Re: bug#14851: linux-libre-3.3.8-gnu disappeared
Date: Fri, 19 Jul 2013 16:50:23 -0300

On Jul 19, 2013, ludo <at> gnu.org (Ludovic Courtès) wrote:

> What do you mean by “multiple GBs of builds per week”?  Linux{,-Libre}
> releases are not that frequent, are they?

They are.  ATM there are 4 active stable releases that each get one new
release per week.  There are 3 other LTS stable branches that get
releases less often.  And then, there are the weekly development -rcs
for the next release, that I seldom start tracking long before the
release is out.  This is just for sources, and it adds up to almost 1GB
per week.

Adding binaries to the picture grows the entire size by a little bit
when it comes to the freesh and planet .debs, and to a larger extent
when it comes to gnewsense/yeeloong (that gets one build per source
release we put out, with a total deb+tar size about the same size of the
source release subdir), and the freed-ora builds (that involve 1-4 rpm
builds per active Fedora release per week, each one weighting some 2GB
total; for most of the time there are 2 or 3 active releases; sometimes
there are 4, depending on whether or not I've already cleaned up the -rc
series for the upcoming release that gets built for the rawhide tree).

> If size turns out to be a problem, we could choose to keep only LTS
> releases on ftp.gnu.org, for instance.

Keeping only LTS wouldn't help guix, though, since you're not using an
LTS release.

-- 
Alexandre Oliva, freedom fighter    http://FSFLA.org/~lxoliva/
You must be the change you wish to see in the world. -- Gandhi
Be Free! -- http://FSFLA.org/   FSF Latin America board member
Free Software Evangelist      Red Hat Brazil Compiler Engineer

Message #35 received at 14851 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Alexandre Oliva <lxoliva <at> fsfla.org>
Cc: 14851 <at> debbugs.gnu.org, Andreas Enge <andreas <at> enge.fr>
Subject: Re: bug#14851: linux-libre-3.3.8-gnu disappeared
Date: Fri, 19 Jul 2013 22:30:50 +0200

Alexandre Oliva <lxoliva <at> fsfla.org> skribis:

> On Jul 19, 2013, ludo <at> gnu.org (Ludovic Courtès) wrote:
>
>> What do you mean by “multiple GBs of builds per week”?  Linux{,-Libre}
>> releases are not that frequent, are they?
>
> They are.  ATM there are 4 active stable releases that each get one new
> release per week.  There are 3 other LTS stable branches that get
> releases less often.  And then, there are the weekly development -rcs
> for the next release, that I seldom start tracking long before the
> release is out.  This is just for sources, and it adds up to almost 1GB
> per week.

Ah OK, I wasn’t aware of that.

> Adding binaries to the picture grows the entire size by a little bit

Yes, but binaries is not what we’re interested in here.  :-)

>> If size turns out to be a problem, we could choose to keep only LTS
>> releases on ftp.gnu.org, for instance.
>
> Keeping only LTS wouldn't help guix, though, since you're not using an
> LTS release.

Heh but that’s not set in stone, and we don’t use Linux-Libre for
anything beyond building libc.

That’ll change in the near future though.

Thanks,
Ludo’.

Message #38 received at 14851 <at> debbugs.gnu.org (full text, mbox):

From: ng0 <contact.ng0 <at> cryptolab.net>
To: 14851 <at> debbugs.gnu.org
Subject: #14851 - linux-libre-3.3.8-gnu disappeared
Date: Mon, 3 Apr 2017 10:53:23 +0000

Andreas, Ludovic, and others:

  This bug report was last modified 3 years and 258 days ago.

It is my understanding that this bug could be closed. I see no more
current problems and it makes no sense to keep the bug open just in case
the problem appears again.

Would you agree?

Message #43 received at 14851-done <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: 14851-done <at> debbugs.gnu.org
Subject: Re: bug#14851: #14851 - linux-libre-3.3.8-gnu disappeared
Date: Fri, 05 May 2017 20:56:43 +0200

ng0 <contact.ng0 <at> cryptolab.net> skribis:

> Andreas, Ludovic, and others:
>
>   This bug report was last modified 3 years and 258 days ago.
>
> It is my understanding that this bug could be closed. I see no more
> current problems and it makes no sense to keep the bug open just in case
> the problem appears again.
>
> Would you agree?

I do!  Especially with the content-addressed mirror we’ve been hosting
with ‘guix publish’.

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.