GNU bug report logs - #25957
gitolite broken: created repositories keep references to /usr/bin for hooks

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 25957 in the body.
You can then email your comments to 25957 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: ng0 <contact.ng0 <at> cryptolab.net>
To: bug-guix <at> gnu.org
Subject: gitolite broken: created repositories keep references to /usr/bin
 for hooks
Date: Fri, 3 Mar 2017 21:58:19 +0000

Our gitolite package currently creates all
(including gitolite-admin.git) git repositories with references to
"/usr/bin/perl" as shebang, which makes it completely useless on
serverside.
Given that the server side in the case of a gitolite from Guix runs an
environment where you will not run perl from /usr/bin/, you will have to
change all hooks manually currently.

When you install perl into the profile of the user which hosts the
repositories and change the shebangs, gitolite can be used.

Message #8 received at 25957 <at> debbugs.gnu.org (full text, mbox):

From: ng0 <contact.ng0 <at> cryptolab.net>
To: 25957 <at> debbugs.gnu.org
Subject: Re: bug#25957: Acknowledgement (gitolite broken: created
 repositories keep references to /usr/bin for hooks)
Date: Fri, 3 Mar 2017 22:27:43 +0000

What makes this worse, with every update (push) of gitolite-admin
repository the shebang of "hooks/update" is reset.
Other repositories seem to keep changes in the hooks shebangs so
far.

Message #11 received at 25957 <at> debbugs.gnu.org (full text, mbox):

From: ng0 <contact.ng0 <at> cryptolab.net>
To: 25957 <at> debbugs.gnu.org
Subject: Re: bug#25957: Acknowledgement (gitolite broken: created
 repositories keep references to /usr/bin for hooks)
Date: Sat, 4 Mar 2017 13:32:42 +0000

On 17-03-03 22:27:43, ng0 wrote:
> What makes this worse, with every update (push) of gitolite-admin
> repository the shebang of "hooks/update" is reset.
> Other repositories seem to keep changes in the hooks shebangs so
> far.
> 
> 
> 

When I build gitolite from guix, this looks trivial to fix.

[user <at> abyayala /gnu/store/jw252kw9blfh1lrrib3yk4fkbj5mvdpm-gitolite-3.6.5/share/gitolite]$ egrep -nr "/usr/"
commands/svnserve:9:$svnserve ||= "/usr/bin/svnserve -r /var/svn/ -t --tunnel-user=%u";
lib/Gitolite/Test/Tsh.pm:42:# path when cwd is [...] at /usr/share/perl5/File/Temp.pm line 902".
lib/Gitolite/Hooks/PostUpdate.pm:62:#!/usr/bin/perl
lib/Gitolite/Hooks/Update.pm:158:#!/usr/bin/perl
lib/Gitolite/Cache.pm:127:    open( REDIS, "|-", "/usr/sbin/redis-server", "-" ) or die "start redis server fail
ed: $!"; 


The parts I want to fix as my immediately affect every user, are in
the directory "lib/Gitolite/Hooks/", I have no idea about redis, but
I think there should be a reference to /gnu/store/ reddis and not
"/usr/sbin/redis-server". Different problem, related bug.. This can be
solved in a commit after this bug.

Message #14 received at 25957 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: ng0 <contact.ng0 <at> cryptolab.net>
Cc: 25957 <at> debbugs.gnu.org
Subject: Re: bug#25957: Acknowledgement (gitolite broken: created
 repositories keep references to /usr/bin for hooks)
Date: Sat, 4 Mar 2017 16:43:09 +0100

Hi ng0,

> I think there should be a reference to /gnu/store/ reddis and not
> "/usr/sbin/redis-server". Different problem, related bug.. This can be
> solved in a commit after this bug.

Yeah.

I would question why a normal application needs to start a redis *server* in the first place. Sounds strange to me. But I agree that if it wants to do that it should use a store reference.

<https://redis.io/topics/admin> says "Redis is designed to be a very long running process in your server" so that definitely reads to me that a normal program shouldn't just start redis-server when it feels like it (and I hope it stops it again later? After reading the source code it doesn't appear that way...).

<http://gitolite.com/gitolite/cache.html> says "WARNING: this has not been tested in a while. YMMV". Uhhhh. Not confidence-inspiring.

Message #17 received at 25957 <at> debbugs.gnu.org (full text, mbox):

From: ng0 <contact.ng0 <at> cryptolab.net>
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 25957 <at> debbugs.gnu.org
Subject: Re: bug#25957: Acknowledgement (gitolite broken: created
 repositories keep references to /usr/bin for hooks)
Date: Sat, 4 Mar 2017 17:33:39 +0000

On 17-03-04 16:43:09, Danny Milosavljevic wrote:
> Hi ng0,
> 
> > I think there should be a reference to /gnu/store/ reddis and not
> > "/usr/sbin/redis-server". Different problem, related bug.. This can be
> > solved in a commit after this bug.
> 
> Yeah.
> 
> I would question why a normal application needs to start a redis *server* in the first place. Sounds strange to me. But I agree that if it wants to do that it should use a store reference.
> 
> <https://redis.io/topics/admin> says "Redis is designed to be a very long running process in your server" so that definitely reads to me that a normal program shouldn't just start redis-server when it feels like it (and I hope it stops it again later? After reading the source code it doesn't appear that way...).
> 
> <http://gitolite.com/gitolite/cache.html> says "WARNING: this has not been tested in a while. YMMV". Uhhhh. Not confidence-inspiring.

It was the first time I read about reddis in gitolite context, and in
all the time I used gitolite I never really needed it when building or
running.
I disregard this as not very important and not really important at all
to fix. It should be fixed in the long run, but my main concern was
usability of gitolite, which has been addressed in one of the two
patches I've sent.

Message #20 received at 25957 <at> debbugs.gnu.org (full text, mbox):

From: zimoun <zimon.toutoune <at> gmail.com>
To: 25957 <at> debbugs.gnu.org
Subject: Re: bug#25957: gitolite broken: created repositories keep
 references to /usr/bin for hooks
Date: Wed, 05 Jan 2022 01:07:07 +0100

Hi,

This old bug [1] is still relevant.

1: <http://issues.guix.gnu.org/issue/25957>

On Sat, 04 Mar 2017 at 13:32, ng0 <contact.ng0 <at> cryptolab.net> wrote:
> On 17-03-03 22:27:43, ng0 wrote:

This previous…

> [user <at> abyayala /gnu/store/jw252kw9blfh1lrrib3yk4fkbj5mvdpm-gitolite-3.6.5/share/gitolite]$ egrep -nr "/usr/"
> commands/svnserve:9:$svnserve ||= "/usr/bin/svnserve -r /var/svn/ -t --tunnel-user=%u";
> lib/Gitolite/Test/Tsh.pm:42:# path when cwd is [...] at /usr/share/perl5/File/Temp.pm line 902".
> lib/Gitolite/Hooks/PostUpdate.pm:62:#!/usr/bin/perl
> lib/Gitolite/Hooks/Update.pm:158:#!/usr/bin/perl
> lib/Gitolite/Cache.pm:127:    open( REDIS, "|-", "/usr/sbin/redis-server", "-" ) or die "start redis server fail
> ed: $!";

…is now…

--8<---------------cut here---------------start------------->8---
share/gitolite/lib/Gitolite/Test/Tsh.pm:42:# path when cwd is [...] at /usr/share/perl5/File/Temp.pm line 902".
share/gitolite/lib/Gitolite/Cache.pm:127:    open( REDIS, "|-", "/usr/sbin/redis-server", "-" ) or die "start redis server failed: $!";
share/gitolite/commands/svnserve:9:$svnserve ||= "/usr/bin/svnserve -r /var/svn/ -t --tunnel-user=%u";
--8<---------------cut here---------------end--------------->8---

…but…

> The parts I want to fix as my immediately affect every user, are in
> the directory "lib/Gitolite/Hooks/", I have no idea about redis, but
> I think there should be a reference to /gnu/store/ reddis and not
> "/usr/sbin/redis-server". Different problem, related bug.. This can be
> solved in a commit after this bug.

…the package redis is not a dependency of gitolite.  Therefore, the
question is: is our Gitolite package working with Redis?  Even using the
/usr/bin one?  Idem for SVN.

Otherwise, I am favor to remove the 2 “problematic” references.   WDYT?


Cheers,
simon

Message #23 received at 25957 <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: zimoun <zimon.toutoune <at> gmail.com>
Cc: 25957 <at> debbugs.gnu.org
Subject: Re: bug#25957: gitolite broken: created repositories keep references
 to /usr/bin for hooks
Date: Wed, 12 Jan 2022 20:07:37 +0200

[Message part 1 (text/plain, inline)]

On Wed, Jan 05, 2022 at 01:07:07AM +0100, zimoun wrote:
> Hi,
> 
> This old bug [1] is still relevant.
> 
> 1: <http://issues.guix.gnu.org/issue/25957>
> 
> On Sat, 04 Mar 2017 at 13:32, ng0 <contact.ng0 <at> cryptolab.net> wrote:
> > On 17-03-03 22:27:43, ng0 wrote:
> 
> This previous…
> 
> > [user <at> abyayala /gnu/store/jw252kw9blfh1lrrib3yk4fkbj5mvdpm-gitolite-3.6.5/share/gitolite]$ egrep -nr "/usr/"
> > commands/svnserve:9:$svnserve ||= "/usr/bin/svnserve -r /var/svn/ -t --tunnel-user=%u";
> > lib/Gitolite/Test/Tsh.pm:42:# path when cwd is [...] at /usr/share/perl5/File/Temp.pm line 902".
> > lib/Gitolite/Hooks/PostUpdate.pm:62:#!/usr/bin/perl
> > lib/Gitolite/Hooks/Update.pm:158:#!/usr/bin/perl
> > lib/Gitolite/Cache.pm:127:    open( REDIS, "|-", "/usr/sbin/redis-server", "-" ) or die "start redis server fail
> > ed: $!";
> 
> …is now…
> 
> --8<---------------cut here---------------start------------->8---
> share/gitolite/lib/Gitolite/Test/Tsh.pm:42:# path when cwd is [...] at /usr/share/perl5/File/Temp.pm line 902".
> share/gitolite/lib/Gitolite/Cache.pm:127:    open( REDIS, "|-", "/usr/sbin/redis-server", "-" ) or die "start redis server failed: $!";
> share/gitolite/commands/svnserve:9:$svnserve ||= "/usr/bin/svnserve -r /var/svn/ -t --tunnel-user=%u";
> --8<---------------cut here---------------end--------------->8---
> 
> …but…
> 
> > The parts I want to fix as my immediately affect every user, are in
> > the directory "lib/Gitolite/Hooks/", I have no idea about redis, but
> > I think there should be a reference to /gnu/store/ reddis and not
> > "/usr/sbin/redis-server". Different problem, related bug.. This can be
> > solved in a commit after this bug.
> 
> …the package redis is not a dependency of gitolite.  Therefore, the
> question is: is our Gitolite package working with Redis?  Even using the
> /usr/bin one?  Idem for SVN.
> 
> Otherwise, I am favor to remove the 2 “problematic” references.   WDYT?

Or change it to search the $PATH for the binary, so it would just be
'redis-server' or 'svnserve'

-- 
Efraim Flashner   <efraim <at> flashner.co.il>   רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[signature.asc (application/pgp-signature, inline)]

Message #26 received at 25957 <at> debbugs.gnu.org (full text, mbox):

From: zimoun <zimon.toutoune <at> gmail.com>
To: Efraim Flashner <efraim <at> flashner.co.il>
Cc: 25957 <at> debbugs.gnu.org
Subject: Re: bug#25957: gitolite broken: created repositories keep
 references to /usr/bin for hooks
Date: Thu, 03 Feb 2022 03:49:13 +0100

Hi Efraim,

On Wed, 12 Jan 2022 at 20:07, Efraim Flashner <efraim <at> flashner.co.il> wrote:

>> …the package redis is not a dependency of gitolite.  Therefore, the
>> question is: is our Gitolite package working with Redis?  Even using the
>> /usr/bin one?  Idem for SVN.
>>
>> Otherwise, I am favor to remove the 2 “problematic” references.   WDYT?
>
> Or change it to search the $PATH for the binary, so it would just be
> 'redis-server' or 'svnserve'

Is our Gitolite package working with Redis?  If not, why try to fix. ;-)


Cheers,
simon

Message #29 received at 25957 <at> debbugs.gnu.org (full text, mbox):

From: zimoun <zimon.toutoune <at> gmail.com>
To: Efraim Flashner <efraim <at> flashner.co.il>
Cc: 25957 <at> debbugs.gnu.org
Subject: Re: bug#25957: gitolite broken: created repositories keep
 references to /usr/bin for hooks
Date: Wed, 23 Mar 2022 11:45:56 +0100

Hi,

On Thu, 03 Feb 2022 at 03:49, zimoun <zimon.toutoune <at> gmail.com> wrote:
> On Wed, 12 Jan 2022 at 20:07, Efraim Flashner <efraim <at> flashner.co.il> wrote:
>
>>> …the package redis is not a dependency of gitolite.  Therefore, the
>>> question is: is our Gitolite package working with Redis?  Even using the
>>> /usr/bin one?  Idem for SVN.
>>>
>>> Otherwise, I am favor to remove the 2 “problematic” references.   WDYT?
>>
>> Or change it to search the $PATH for the binary, so it would just be
>> 'redis-server' or 'svnserve'
>
> Is our Gitolite package working with Redis?  If not, why try to fix. ;-)

What is the status of this old bug [1]?  Is it actionable?  If yes, what
is the action?  If no, let close it. :-)  WDYT?


1: <http://issues.guix.gnu.org/issue/25957>


Cheers,
simon

Message #32 received at 25957 <at> debbugs.gnu.org (full text, mbox):

From: Maxime Devos <maximedevos <at> telenet.be>
To: zimoun <zimon.toutoune <at> gmail.com>, Efraim Flashner <efraim <at> flashner.co.il>
Cc: 25957 <at> debbugs.gnu.org
Subject: Re: bug#25957: gitolite broken: created repositories keep
 references to /usr/bin for hooks
Date: Wed, 23 Mar 2022 13:44:29 +0100

[Message part 1 (text/plain, inline)]

zimoun schreef op wo 23-03-2022 om 11:45 [+0100]:
> > On Wed, 12 Jan 2022 at 20:07, Efraim Flashner
> > <efraim <at> flashner.co.il> wrote:
> > 
> > > > …the package redis is not a dependency of gitolite.  Therefore,
> > > > the
> > > > question is: is our Gitolite package working with Redis?  Even
> > > > using the
> > > > /usr/bin one?  Idem for SVN.
> > > > 
> > > > Otherwise, I am favor to remove the 2 “problematic”
> > > > references.   WDYT?
> > > 
> > > Or change it to search the $PATH for the binary, so it would just
> > > be
> > > 'redis-server' or 'svnserve'
> > 
> > Is our Gitolite package working with Redis?  If not, why try to
> > fix. ;-)
> 
> What is the status of this old bug [1]?  Is it actionable?  If yes,
> what
> is the action?  If no, let close it. :-)  WDYT?

Seems like all we have to do is 'substitute*' a '/usr/bin/svnserve'
into a '/gnu/store/...' (untested), so seems actionable to me.
Alternatively, as Efraim wrote, let it search the $PATH (that might be
useful if adding svnserve would increase the closure too much and it is
an optional dependency in practice?).

Greetings,
Maxime.

[signature.asc (application/pgp-signature, inline)]

Message #35 received at 25957 <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: Maxime Devos <maximedevos <at> telenet.be>
Cc: 25957 <at> debbugs.gnu.org, zimoun <zimon.toutoune <at> gmail.com>
Subject: Re: bug#25957: gitolite broken: created repositories keep references
 to /usr/bin for hooks
Date: Mon, 28 Mar 2022 09:49:29 +0300

[Message part 1 (text/plain, inline)]

On Wed, Mar 23, 2022 at 01:44:29PM +0100, Maxime Devos wrote:
> zimoun schreef op wo 23-03-2022 om 11:45 [+0100]:
> > > On Wed, 12 Jan 2022 at 20:07, Efraim Flashner
> > > <efraim <at> flashner.co.il> wrote:
> > > 
> > > > > …the package redis is not a dependency of gitolite.  Therefore,
> > > > > the
> > > > > question is: is our Gitolite package working with Redis?  Even
> > > > > using the
> > > > > /usr/bin one?  Idem for SVN.
> > > > > 
> > > > > Otherwise, I am favor to remove the 2 “problematic”
> > > > > references.   WDYT?
> > > > 
> > > > Or change it to search the $PATH for the binary, so it would just
> > > > be
> > > > 'redis-server' or 'svnserve'
> > > 
> > > Is our Gitolite package working with Redis?  If not, why try to
> > > fix. ;-)
> > 
> > What is the status of this old bug [1]?  Is it actionable?  If yes,
> > what
> > is the action?  If no, let close it. :-)  WDYT?
> 
> Seems like all we have to do is 'substitute*' a '/usr/bin/svnserve'
> into a '/gnu/store/...' (untested), so seems actionable to me.
> Alternatively, as Efraim wrote, let it search the $PATH (that might be
> useful if adding svnserve would increase the closure too much and it is
> an optional dependency in practice?).

I spent some time looking at gitolite and the service. As I understand
it, with the exception of svnserve, it searches $PATH for a number of
different binaries, including git-annex. I believe that this would only
work if git-annex (and potentially other packages) are installed
globally.

In addition, git (not git-minimal) and openssh are propagated inputs AND
wrapped. I haven't tested to see if wrapping only is enough.

I think the best choice is to:
A: Replace /usr/bin/svnserve with svnserve so it will just search $PATH,
like it does with the other helpers.
B: Adjust the service so that it automatically creates a variant (or
just a wrapped version) of the package which is wrapped with a list of
additional packages so that they can be in gitolite's path. If I were
deploying this to an arm device I wouldn't want it wrapped with
git-annex since it doesn't build, but would definitely want it for an
x86_64 machine.

I suppose we should try to find someone who is using the gitolite
service and see if they can be our test subject for wrapping the package
with optional addons.

-- 
Efraim Flashner   <efraim <at> flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[signature.asc (application/pgp-signature, inline)]

Message #38 received at 25957 <at> debbugs.gnu.org (full text, mbox):

From: "Thompson, David" <dthompson2 <at> worcester.edu>
To: Efraim Flashner <efraim <at> flashner.co.il>,
 Maxime Devos <maximedevos <at> telenet.be>, 
 zimoun <zimon.toutoune <at> gmail.com>, 25957 <at> debbugs.gnu.org
Subject: Re: [EXT] bug#25957: gitolite broken: created repositories keep
 references to /usr/bin for hooks
Date: Thu, 1 Sep 2022 09:59:55 -0400

Hi all,

Reviving this old thread.

On Mon, Mar 28, 2022 at 2:51 AM Efraim Flashner <efraim <at> flashner.co.il> wrote:
> >
> > Seems like all we have to do is 'substitute*' a '/usr/bin/svnserve'
> > into a '/gnu/store/...' (untested), so seems actionable to me.
> > Alternatively, as Efraim wrote, let it search the $PATH (that might be
> > useful if adding svnserve would increase the closure too much and it is
> > an optional dependency in practice?).
>
> I spent some time looking at gitolite and the service. As I understand
> it, with the exception of svnserve, it searches $PATH for a number of
> different binaries, including git-annex. I believe that this would only
> work if git-annex (and potentially other packages) are installed
> globally.
>
> In addition, git (not git-minimal) and openssh are propagated inputs AND
> wrapped. I haven't tested to see if wrapping only is enough.
>
> I think the best choice is to:
> A: Replace /usr/bin/svnserve with svnserve so it will just search $PATH,
> like it does with the other helpers.

I see that you have done this.  Thanks!  We could also replace the
reference to /usr/sbin/redis-server in src/lib/Gitolite/Cache.pm.
That's the only other /usr reference I can find (that isn't in a
comment) in the output.  I have the patch ready if that sounds good to
you.

> B: Adjust the service so that it automatically creates a variant (or
> just a wrapped version) of the package which is wrapped with a list of
> additional packages so that they can be in gitolite's path. If I were
> deploying this to an arm device I wouldn't want it wrapped with
> git-annex since it doesn't build, but would definitely want it for an
> x86_64 machine.

The service configuration record could accept a list of addons like
'(git-annex cache svnserve), with a default of no addons '(), and
create a package that extends the gitolite package with the
appropriate propagated inputs.  Does that sound like what you had in
mind?  A more robust solution could modify the build to hardcode the
store paths needed for the add-ons but given that we already propagate
git and openssh I don't think it's necessary right now.

> I suppose we should try to find someone who is using the gitolite
> service and see if they can be our test subject for wrapping the package
> with optional addons.

I use the gitolite service and can be the test subject.  I don't
currently use any add-ons, but the redis one sounds easy enough to try
and hey maybe it's a good excuse to finally learn how to use
git-annex.

As a longer term thing, it would be cool to revisit propagating git
and openssh in this package.  I punted on it back in 2015 for the
reason stated in the source comments but maybe there's a reasonable
and reliable way to directly embed the store paths now.

- Dave

Message #41 received at 25957 <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: "Thompson, David" <dthompson2 <at> worcester.edu>
Cc: 25957 <at> debbugs.gnu.org, Maxime Devos <maximedevos <at> telenet.be>,
 zimoun <zimon.toutoune <at> gmail.com>
Subject: Re: [EXT] bug#25957: gitolite broken: created repositories keep
 references to /usr/bin for hooks
Date: Thu, 1 Sep 2022 17:20:01 +0300

[Message part 1 (text/plain, inline)]

On Thu, Sep 01, 2022 at 09:59:55AM -0400, Thompson, David wrote:
> Hi all,
> 
> Reviving this old thread.
> 
> On Mon, Mar 28, 2022 at 2:51 AM Efraim Flashner <efraim <at> flashner.co.il> wrote:
> > >
> > > Seems like all we have to do is 'substitute*' a '/usr/bin/svnserve'
> > > into a '/gnu/store/...' (untested), so seems actionable to me.
> > > Alternatively, as Efraim wrote, let it search the $PATH (that might be
> > > useful if adding svnserve would increase the closure too much and it is
> > > an optional dependency in practice?).
> >
> > I spent some time looking at gitolite and the service. As I understand
> > it, with the exception of svnserve, it searches $PATH for a number of
> > different binaries, including git-annex. I believe that this would only
> > work if git-annex (and potentially other packages) are installed
> > globally.
> >
> > In addition, git (not git-minimal) and openssh are propagated inputs AND
> > wrapped. I haven't tested to see if wrapping only is enough.
> >
> > I think the best choice is to:
> > A: Replace /usr/bin/svnserve with svnserve so it will just search $PATH,
> > like it does with the other helpers.
> 
> I see that you have done this.  Thanks!  We could also replace the
> reference to /usr/sbin/redis-server in src/lib/Gitolite/Cache.pm.
> That's the only other /usr reference I can find (that isn't in a
> comment) in the output.  I have the patch ready if that sounds good to
> you.

Sounds good to me

> > B: Adjust the service so that it automatically creates a variant (or
> > just a wrapped version) of the package which is wrapped with a list of
> > additional packages so that they can be in gitolite's path. If I were
> > deploying this to an arm device I wouldn't want it wrapped with
> > git-annex since it doesn't build, but would definitely want it for an
> > x86_64 machine.
> 
> The service configuration record could accept a list of addons like
> '(git-annex cache svnserve), with a default of no addons '(), and
> create a package that extends the gitolite package with the
> appropriate propagated inputs.  Does that sound like what you had in
> mind?  A more robust solution could modify the build to hardcode the
> store paths needed for the add-ons but given that we already propagate
> git and openssh I don't think it's necessary right now.

Assuming this is deployed into some sort of container then propagated
inputs wouldn't help much, we'd need either the PATH for the container
to be extended to include those extra packages or to have gitolite
itself wrapped similar to icedove/wayland. Just extending the PATH in
the #:environment-variables would be enough I'd think.

> > I suppose we should try to find someone who is using the gitolite
> > service and see if they can be our test subject for wrapping the package
> > with optional addons.
> 
> I use the gitolite service and can be the test subject.  I don't
> currently use any add-ons, but the redis one sounds easy enough to try
> and hey maybe it's a good excuse to finally learn how to use
> git-annex.
> 
> As a longer term thing, it would be cool to revisit propagating git
> and openssh in this package.  I punted on it back in 2015 for the
> reason stated in the source comments but maybe there's a reasonable
> and reliable way to directly embed the store paths now.

It's actually been forever since I looked at gitolite so I don't know
remember what those inputs were needed for, but it'd be great to improve
the service.

> - Dave

Interestingly, I almost have a working ghc-8.6 for aarch64 after all
these years.

-- 
Efraim Flashner   <efraim <at> flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[signature.asc (application/pgp-signature, inline)]

Message #44 received at 25957 <at> debbugs.gnu.org (full text, mbox):

From: "Thompson, David" <dthompson2 <at> worcester.edu>
To: Efraim Flashner <efraim <at> flashner.co.il>, "Thompson,
 David" <dthompson2 <at> worcester.edu>, 
 Maxime Devos <maximedevos <at> telenet.be>, zimoun <zimon.toutoune <at> gmail.com>,
 25957 <at> debbugs.gnu.org
Subject: Re: [EXT] Re: [EXT] bug#25957: gitolite broken: created repositories
 keep references to /usr/bin for hooks
Date: Thu, 1 Sep 2022 10:41:21 -0400

Hi Efraim,

On Thu, Sep 1, 2022 at 10:20 AM Efraim Flashner <efraim <at> flashner.co.il> wrote:
>
> On Thu, Sep 01, 2022 at 09:59:55AM -0400, Thompson, David wrote:
> > Hi all,
> >
> > Reviving this old thread.
> >
> > On Mon, Mar 28, 2022 at 2:51 AM Efraim Flashner <efraim <at> flashner.co.il> wrote:
> > > >
> > > > Seems like all we have to do is 'substitute*' a '/usr/bin/svnserve'
> > > > into a '/gnu/store/...' (untested), so seems actionable to me.
> > > > Alternatively, as Efraim wrote, let it search the $PATH (that might be
> > > > useful if adding svnserve would increase the closure too much and it is
> > > > an optional dependency in practice?).
> > >
> > > I spent some time looking at gitolite and the service. As I understand
> > > it, with the exception of svnserve, it searches $PATH for a number of
> > > different binaries, including git-annex. I believe that this would only
> > > work if git-annex (and potentially other packages) are installed
> > > globally.
> > >
> > > In addition, git (not git-minimal) and openssh are propagated inputs AND
> > > wrapped. I haven't tested to see if wrapping only is enough.
> > >
> > > I think the best choice is to:
> > > A: Replace /usr/bin/svnserve with svnserve so it will just search $PATH,
> > > like it does with the other helpers.
> >
> > I see that you have done this.  Thanks!  We could also replace the
> > reference to /usr/sbin/redis-server in src/lib/Gitolite/Cache.pm.
> > That's the only other /usr reference I can find (that isn't in a
> > comment) in the output.  I have the patch ready if that sounds good to
> > you.
>
> Sounds good to me

Thanks, pushed as commit c053dfa52dc778eb3d965f58a85c435ae7fab0dd.

> > > B: Adjust the service so that it automatically creates a variant (or
> > > just a wrapped version) of the package which is wrapped with a list of
> > > additional packages so that they can be in gitolite's path. If I were
> > > deploying this to an arm device I wouldn't want it wrapped with
> > > git-annex since it doesn't build, but would definitely want it for an
> > > x86_64 machine.
> >
> > The service configuration record could accept a list of addons like
> > '(git-annex cache svnserve), with a default of no addons '(), and
> > create a package that extends the gitolite package with the
> > appropriate propagated inputs.  Does that sound like what you had in
> > mind?  A more robust solution could modify the build to hardcode the
> > store paths needed for the add-ons but given that we already propagate
> > git and openssh I don't think it's necessary right now.
>
> Assuming this is deployed into some sort of container then propagated
> inputs wouldn't help much, we'd need either the PATH for the container
> to be extended to include those extra packages or to have gitolite
> itself wrapped similar to icedove/wayland. Just extending the PATH in
> the #:environment-variables would be enough I'd think.

Hmm, I hadn't thought about the container use case.  Your approach
sounds like the way to go.  For what it's worth, I think the gitolite
service as-is would suffer the same issue in a containerized
environment because it relies upon the git and openssh propagated
inputs to do anything at all.  With the gitolite service in my system,
/run/current-system/profile/bin has both git and ssh in it due to the
propagation.  So it sounds like there's 2 steps needed: 1) Use a
wrapper like icedove/wayland for the base gitolite package so that git
and openssh no longer need propagation, and then 2) extend the
gitolite service to wrap up additional packages needed for the desired
extensions.  Sound good?

> > > I suppose we should try to find someone who is using the gitolite
> > > service and see if they can be our test subject for wrapping the package
> > > with optional addons.
> >
> > I use the gitolite service and can be the test subject.  I don't
> > currently use any add-ons, but the redis one sounds easy enough to try
> > and hey maybe it's a good excuse to finally learn how to use
> > git-annex.
> >
> > As a longer term thing, it would be cool to revisit propagating git
> > and openssh in this package.  I punted on it back in 2015 for the
> > reason stated in the source comments but maybe there's a reasonable
> > and reliable way to directly embed the store paths now.
>
> It's actually been forever since I looked at gitolite so I don't know
> remember what those inputs were needed for, but it'd be great to improve
> the service.

Are you referring to git and openssh or redis, svnserve, git-annex,
etc.?  I'm no expert and I really don't like Perl, but I know gitolite
well enough to explain some of this stuff.

> Interestingly, I almost have a working ghc-8.6 for aarch64 after all
> these years.

Some things move at a glacial pace, but eventually they get done.
Best of luck wrapping that up.  :)

- Dave

Message #47 received at 25957 <at> debbugs.gnu.org (full text, mbox):

From: zimoun <zimon.toutoune <at> gmail.com>
To: "Thompson, David" <dthompson2 <at> worcester.edu>, Efraim Flashner
 <efraim <at> flashner.co.il>, "Thompson,
 David" <dthompson2 <at> worcester.edu>, Maxime Devos <maximedevos <at> telenet.be>,
 25957 <at> debbugs.gnu.org
Subject: Re: bug#25957: [EXT] Re: [EXT] bug#25957: gitolite broken: created
 repositories keep references to /usr/bin for hooks
Date: Thu, 01 Sep 2022 19:00:45 +0200

Hi,

On jeu., 01 sept. 2022 at 10:41, "Thompson, David" <dthompson2 <at> worcester.edu> wrote:

> Thanks, pushed as commit c053dfa52dc778eb3d965f58a85c435ae7fab0dd.

Cool!  Thank you.


Cheers,
simon

Message #50 received at 25957 <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: "Thompson, David" <dthompson2 <at> worcester.edu>
Cc: 25957 <at> debbugs.gnu.org, Maxime Devos <maximedevos <at> telenet.be>,
 zimoun <zimon.toutoune <at> gmail.com>
Subject: Re: [EXT] Re: [EXT] bug#25957: gitolite broken: created repositories
 keep references to /usr/bin for hooks
Date: Fri, 2 Sep 2022 09:56:49 +0300

[Message part 1 (text/plain, inline)]

On Thu, Sep 01, 2022 at 10:41:21AM -0400, Thompson, David wrote:
> Hi Efraim,
> 
> On Thu, Sep 1, 2022 at 10:20 AM Efraim Flashner <efraim <at> flashner.co.il> wrote:
> >
> > On Thu, Sep 01, 2022 at 09:59:55AM -0400, Thompson, David wrote:
> > > Hi all,
> > >
> > > Reviving this old thread.
> > >
> > > On Mon, Mar 28, 2022 at 2:51 AM Efraim Flashner <efraim <at> flashner.co.il> wrote:
> > > > >
> > > > > Seems like all we have to do is 'substitute*' a '/usr/bin/svnserve'
> > > > > into a '/gnu/store/...' (untested), so seems actionable to me.
> > > > > Alternatively, as Efraim wrote, let it search the $PATH (that might be
> > > > > useful if adding svnserve would increase the closure too much and it is
> > > > > an optional dependency in practice?).
> > > >
> > > > I spent some time looking at gitolite and the service. As I understand
> > > > it, with the exception of svnserve, it searches $PATH for a number of
> > > > different binaries, including git-annex. I believe that this would only
> > > > work if git-annex (and potentially other packages) are installed
> > > > globally.
> > > >
> > > > In addition, git (not git-minimal) and openssh are propagated inputs AND
> > > > wrapped. I haven't tested to see if wrapping only is enough.
> > > >
> > > > I think the best choice is to:
> > > > A: Replace /usr/bin/svnserve with svnserve so it will just search $PATH,
> > > > like it does with the other helpers.
> > >
> > > I see that you have done this.  Thanks!  We could also replace the
> > > reference to /usr/sbin/redis-server in src/lib/Gitolite/Cache.pm.
> > > That's the only other /usr reference I can find (that isn't in a
> > > comment) in the output.  I have the patch ready if that sounds good to
> > > you.
> >
> > Sounds good to me
> 
> Thanks, pushed as commit c053dfa52dc778eb3d965f58a85c435ae7fab0dd.
> 
> > > > B: Adjust the service so that it automatically creates a variant (or
> > > > just a wrapped version) of the package which is wrapped with a list of
> > > > additional packages so that they can be in gitolite's path. If I were
> > > > deploying this to an arm device I wouldn't want it wrapped with
> > > > git-annex since it doesn't build, but would definitely want it for an
> > > > x86_64 machine.
> > >
> > > The service configuration record could accept a list of addons like
> > > '(git-annex cache svnserve), with a default of no addons '(), and
> > > create a package that extends the gitolite package with the
> > > appropriate propagated inputs.  Does that sound like what you had in
> > > mind?  A more robust solution could modify the build to hardcode the
> > > store paths needed for the add-ons but given that we already propagate
> > > git and openssh I don't think it's necessary right now.
> >
> > Assuming this is deployed into some sort of container then propagated
> > inputs wouldn't help much, we'd need either the PATH for the container
> > to be extended to include those extra packages or to have gitolite
> > itself wrapped similar to icedove/wayland. Just extending the PATH in
> > the #:environment-variables would be enough I'd think.
> 
> Hmm, I hadn't thought about the container use case.  Your approach
> sounds like the way to go.  For what it's worth, I think the gitolite
> service as-is would suffer the same issue in a containerized
> environment because it relies upon the git and openssh propagated
> inputs to do anything at all.  With the gitolite service in my system,
> /run/current-system/profile/bin has both git and ssh in it due to the
> propagation.  So it sounds like there's 2 steps needed: 1) Use a
> wrapper like icedove/wayland for the base gitolite package so that git
> and openssh no longer need propagation, and then 2) extend the
> gitolite service to wrap up additional packages needed for the desired
> extensions.  Sound good?
> 
> > > > I suppose we should try to find someone who is using the gitolite
> > > > service and see if they can be our test subject for wrapping the package
> > > > with optional addons.
> > >
> > > I use the gitolite service and can be the test subject.  I don't
> > > currently use any add-ons, but the redis one sounds easy enough to try
> > > and hey maybe it's a good excuse to finally learn how to use
> > > git-annex.
> > >
> > > As a longer term thing, it would be cool to revisit propagating git
> > > and openssh in this package.  I punted on it back in 2015 for the
> > > reason stated in the source comments but maybe there's a reasonable
> > > and reliable way to directly embed the store paths now.
> >
> > It's actually been forever since I looked at gitolite so I don't know
> > remember what those inputs were needed for, but it'd be great to improve
> > the service.
> 
> Are you referring to git and openssh or redis, svnserve, git-annex,
> etc.?  I'm no expert and I really don't like Perl, but I know gitolite
> well enough to explain some of this stuff.
> 
> > Interestingly, I almost have a working ghc-8.6 for aarch64 after all
> > these years.
> 
> Some things move at a glacial pace, but eventually they get done.
> Best of luck wrapping that up.  :)

I took a look at the gitolite service finally and I hadn't realized
there wasn't a running daemon to containerize. I assumed we could do
something like:

(start $~(make-forkexec-constructor/container
            (list ...)
            #:environment-variables
            '("PATH=...")
            #:mappings ...))

Given that's not the case then I'd need to look at gitolite itself to
see how it calls the other binaries it expects to be available, and if
wrapping it would be enough or if we would need to just propagate the
other packages for functionality.

-- 
Efraim Flashner   <efraim <at> flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[signature.asc (application/pgp-signature, inline)]

Message #53 received at 25957 <at> debbugs.gnu.org (full text, mbox):

From: "Thompson, David" <dthompson2 <at> worcester.edu>
To: Efraim Flashner <efraim <at> flashner.co.il>, "Thompson,
 David" <dthompson2 <at> worcester.edu>, 
 zimoun <zimon.toutoune <at> gmail.com>, 25957 <at> debbugs.gnu.org
Subject: Re: bug#25957: gitolite broken: created repositories keep references
 to /usr/bin for hooks
Date: Fri, 2 Sep 2022 07:11:54 -0400

On Fri, Sep 2, 2022 at 3:00 AM Efraim Flashner <efraim <at> flashner.co.il> wrote:
>
> I took a look at the gitolite service finally and I hadn't realized
> there wasn't a running daemon to containerize. I assumed we could do
> something like:
>
> (start $~(make-forkexec-constructor/container
>             (list ...)
>             #:environment-variables
>             '("PATH=...")
>             #:mappings ...))
>
> Given that's not the case then I'd need to look at gitolite itself to
> see how it calls the other binaries it expects to be available, and if
> wrapping it would be enough or if we would need to just propagate the
> other packages for functionality.

Gitolite simply expects tools like git to be on $PATH.  It's a pretty
naive system, there's nothing like a configure script that is
determining the absolute file name of these tools and substituting
those names into the built files.

The executable is already wrapped so that coreutils, findutils, and
git are on $PATH, but notably not openssh:

(add-after 'install 'wrap-scripts
                    (lambda* (#:key inputs outputs #:allow-other-keys)
                      (let ((out (assoc-ref outputs "out"))
                            (coreutils (assoc-ref inputs "coreutils"))
                            (findutils (assoc-ref inputs "findutils"))
                            (git (assoc-ref inputs "git")))
                        (wrap-program (string-append out "/bin/gitolite")
                          `("PATH" ":" prefix
                            ,(map (lambda (dir)
                                    (string-append dir "/bin"))
                                  (list out coreutils findutils git)))))))

However, git and openssh are still propagated inputs. I'm going to
move the propagated inputs to regular inputs, potentially add openssh
to the wrapper once I remind myself what gitolite does with those
tools, and test it all out on my server using the gitolite service.
If that all works, we have a good starting point for adding extension
support in the service.

- Dave

Message #56 received at 25957 <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: "Thompson, David" <dthompson2 <at> worcester.edu>
Cc: 25957 <at> debbugs.gnu.org, zimoun <zimon.toutoune <at> gmail.com>
Subject: Re: bug#25957: gitolite broken: created repositories keep references
 to /usr/bin for hooks
Date: Fri, 2 Sep 2022 15:41:42 +0300

[Message part 1 (text/plain, inline)]

On Fri, Sep 02, 2022 at 07:11:54AM -0400, Thompson, David wrote:
> On Fri, Sep 2, 2022 at 3:00 AM Efraim Flashner <efraim <at> flashner.co.il> wrote:
> >
> > I took a look at the gitolite service finally and I hadn't realized
> > there wasn't a running daemon to containerize. I assumed we could do
> > something like:
> >
> > (start $~(make-forkexec-constructor/container
> >             (list ...)
> >             #:environment-variables
> >             '("PATH=...")
> >             #:mappings ...))
> >
> > Given that's not the case then I'd need to look at gitolite itself to
> > see how it calls the other binaries it expects to be available, and if
> > wrapping it would be enough or if we would need to just propagate the
> > other packages for functionality.
> 
> Gitolite simply expects tools like git to be on $PATH.  It's a pretty
> naive system, there's nothing like a configure script that is
> determining the absolute file name of these tools and substituting
> those names into the built files.
> 
> The executable is already wrapped so that coreutils, findutils, and
> git are on $PATH, but notably not openssh:
> 
> (add-after 'install 'wrap-scripts
>                     (lambda* (#:key inputs outputs #:allow-other-keys)
>                       (let ((out (assoc-ref outputs "out"))
>                             (coreutils (assoc-ref inputs "coreutils"))
>                             (findutils (assoc-ref inputs "findutils"))
>                             (git (assoc-ref inputs "git")))
>                         (wrap-program (string-append out "/bin/gitolite")
>                           `("PATH" ":" prefix
>                             ,(map (lambda (dir)
>                                     (string-append dir "/bin"))
>                                   (list out coreutils findutils git)))))))
> 
> However, git and openssh are still propagated inputs. I'm going to
> move the propagated inputs to regular inputs, potentially add openssh
> to the wrapper once I remind myself what gitolite does with those
> tools, and test it all out on my server using the gitolite service.
> If that all works, we have a good starting point for adding extension
> support in the service.

I like it. Let us know how it goes.

-- 
Efraim Flashner   <efraim <at> flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[signature.asc (application/pgp-signature, inline)]

Message #59 received at 25957 <at> debbugs.gnu.org (full text, mbox):

From: "Thompson, David" <dthompson2 <at> worcester.edu>
To: Efraim Flashner <efraim <at> flashner.co.il>, "Thompson,
 David" <dthompson2 <at> worcester.edu>, 
 zimoun <zimon.toutoune <at> gmail.com>, 25957 <at> debbugs.gnu.org
Subject: Re: [EXT] Re: bug#25957: gitolite broken: created repositories keep
 references to /usr/bin for hooks
Date: Fri, 2 Sep 2022 08:50:21 -0400

On Fri, Sep 2, 2022 at 8:44 AM Efraim Flashner <efraim <at> flashner.co.il> wrote:
>
> On Fri, Sep 02, 2022 at 07:11:54AM -0400, Thompson, David wrote:
> > On Fri, Sep 2, 2022 at 3:00 AM Efraim Flashner <efraim <at> flashner.co.il> wrote:
> > >
> > > I took a look at the gitolite service finally and I hadn't realized
> > > there wasn't a running daemon to containerize. I assumed we could do
> > > something like:
> > >
> > > (start $~(make-forkexec-constructor/container
> > >             (list ...)
> > >             #:environment-variables
> > >             '("PATH=...")
> > >             #:mappings ...))
> > >
> > > Given that's not the case then I'd need to look at gitolite itself to
> > > see how it calls the other binaries it expects to be available, and if
> > > wrapping it would be enough or if we would need to just propagate the
> > > other packages for functionality.
> >
> > Gitolite simply expects tools like git to be on $PATH.  It's a pretty
> > naive system, there's nothing like a configure script that is
> > determining the absolute file name of these tools and substituting
> > those names into the built files.
> >
> > The executable is already wrapped so that coreutils, findutils, and
> > git are on $PATH, but notably not openssh:
> >
> > (add-after 'install 'wrap-scripts
> >                     (lambda* (#:key inputs outputs #:allow-other-keys)
> >                       (let ((out (assoc-ref outputs "out"))
> >                             (coreutils (assoc-ref inputs "coreutils"))
> >                             (findutils (assoc-ref inputs "findutils"))
> >                             (git (assoc-ref inputs "git")))
> >                         (wrap-program (string-append out "/bin/gitolite")
> >                           `("PATH" ":" prefix
> >                             ,(map (lambda (dir)
> >                                     (string-append dir "/bin"))
> >                                   (list out coreutils findutils git)))))))
> >
> > However, git and openssh are still propagated inputs. I'm going to
> > move the propagated inputs to regular inputs, potentially add openssh
> > to the wrapper once I remind myself what gitolite does with those
> > tools, and test it all out on my server using the gitolite service.
> > If that all works, we have a good starting point for adding extension
> > support in the service.
>
> I like it. Let us know how it goes.

The problem is that gitolite generates git hooks for the repositories
that it manages, and those hooks invoke git, so the only way those
scripts will be able to work (without input propagation) is to find a
way to inject the proper PATH or find a way to replace references to
things like 'git diff' with '/gnu/store/.../git diff'.  I'm going to
keep exploring and report back when I have something to show.

- Dave

Message #62 received at 25957 <at> debbugs.gnu.org (full text, mbox):

From: "Thompson, David" <dthompson2 <at> worcester.edu>
To: Efraim Flashner <efraim <at> flashner.co.il>, "Thompson,
 David" <dthompson2 <at> worcester.edu>, 
 zimoun <zimon.toutoune <at> gmail.com>, 25957 <at> debbugs.gnu.org
Subject: Re: bug#25957: gitolite broken: created repositories keep references
 to /usr/bin for hooks
Date: Fri, 2 Sep 2022 15:58:09 -0400

[Message part 1 (text/plain, inline)]

On Fri, Sep 2, 2022 at 8:50 AM Thompson, David <dthompson2 <at> worcester.edu> wrote:
>
> On Fri, Sep 2, 2022 at 8:44 AM Efraim Flashner <efraim <at> flashner.co.il> wrote:
> >
> > On Fri, Sep 02, 2022 at 07:11:54AM -0400, Thompson, David wrote:
> > > On Fri, Sep 2, 2022 at 3:00 AM Efraim Flashner <efraim <at> flashner.co.il> wrote:
> > > >
> > > > I took a look at the gitolite service finally and I hadn't realized
> > > > there wasn't a running daemon to containerize. I assumed we could do
> > > > something like:
> > > >
> > > > (start $~(make-forkexec-constructor/container
> > > >             (list ...)
> > > >             #:environment-variables
> > > >             '("PATH=...")
> > > >             #:mappings ...))
> > > >
> > > > Given that's not the case then I'd need to look at gitolite itself to
> > > > see how it calls the other binaries it expects to be available, and if
> > > > wrapping it would be enough or if we would need to just propagate the
> > > > other packages for functionality.
> > >
> > > Gitolite simply expects tools like git to be on $PATH.  It's a pretty
> > > naive system, there's nothing like a configure script that is
> > > determining the absolute file name of these tools and substituting
> > > those names into the built files.
> > >
> > > The executable is already wrapped so that coreutils, findutils, and
> > > git are on $PATH, but notably not openssh:
> > >
> > > (add-after 'install 'wrap-scripts
> > >                     (lambda* (#:key inputs outputs #:allow-other-keys)
> > >                       (let ((out (assoc-ref outputs "out"))
> > >                             (coreutils (assoc-ref inputs "coreutils"))
> > >                             (findutils (assoc-ref inputs "findutils"))
> > >                             (git (assoc-ref inputs "git")))
> > >                         (wrap-program (string-append out "/bin/gitolite")
> > >                           `("PATH" ":" prefix
> > >                             ,(map (lambda (dir)
> > >                                     (string-append dir "/bin"))
> > >                                   (list out coreutils findutils git)))))))
> > >
> > > However, git and openssh are still propagated inputs. I'm going to
> > > move the propagated inputs to regular inputs, potentially add openssh
> > > to the wrapper once I remind myself what gitolite does with those
> > > tools, and test it all out on my server using the gitolite service.
> > > If that all works, we have a good starting point for adding extension
> > > support in the service.
> >
> > I like it. Let us know how it goes.
>
> The problem is that gitolite generates git hooks for the repositories
> that it manages, and those hooks invoke git, so the only way those
> scripts will be able to work (without input propagation) is to find a
> way to inject the proper PATH or find a way to replace references to
> things like 'git diff' with '/gnu/store/.../git diff'.  I'm going to
> keep exploring and report back when I have something to show.

After several rounds of experimentation and breaking my git server a
few times, here's what I've found:

* Changing git and openssh to be regular inputs and wrapping both
gitolite and gitolite-shell with a $PATH that contains git works and
it's very little extra code.

* Trying to replace every invocation of a git command took a lot of
grepping and crafting of regexps to use for substitute* and I never
got to a point where the result wasn't buggy.  In particular,
gitolite-shell never worked properly so I couldn't push to my repos.

So, I think the simple wrapper approach is the way to go.  Patch
attached.  I tested on my git server by making changes to my gitolite
configuration and pushing those changes to the special gitolite-admin
repo.  This causes gitolite to refresh internal configuration using a
git hook, so I know that hooks can find the executables they need.
That plus the 'gitolite setup' invocation made by the service
activation script covers a fair amount of surface area, so I feel
comfortable committing it.  What do you think?

Once this part is done, I'll turn my attention to the optional extensions.

- Dave

[0001-gnu-gitolite-Wrap-programs-instead-of-using-propagat.patch (text/x-patch, attachment)]

Message #65 received at 25957 <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: "Thompson, David" <dthompson2 <at> worcester.edu>
Cc: 25957 <at> debbugs.gnu.org, zimoun <zimon.toutoune <at> gmail.com>
Subject: Re: bug#25957: gitolite broken: created repositories keep references
 to /usr/bin for hooks
Date: Sun, 4 Sep 2022 10:26:05 +0300

[Message part 1 (text/plain, inline)]

On Fri, Sep 02, 2022 at 03:58:09PM -0400, Thompson, David wrote:
> On Fri, Sep 2, 2022 at 8:50 AM Thompson, David <dthompson2 <at> worcester.edu> wrote:
> >
> > On Fri, Sep 2, 2022 at 8:44 AM Efraim Flashner <efraim <at> flashner.co.il> wrote:
> > >
> > > On Fri, Sep 02, 2022 at 07:11:54AM -0400, Thompson, David wrote:
> > > > On Fri, Sep 2, 2022 at 3:00 AM Efraim Flashner <efraim <at> flashner.co.il> wrote:
> > > > >
> > > > > I took a look at the gitolite service finally and I hadn't realized
> > > > > there wasn't a running daemon to containerize. I assumed we could do
> > > > > something like:
> > > > >
> > > > > (start $~(make-forkexec-constructor/container
> > > > >             (list ...)
> > > > >             #:environment-variables
> > > > >             '("PATH=...")
> > > > >             #:mappings ...))
> > > > >
> > > > > Given that's not the case then I'd need to look at gitolite itself to
> > > > > see how it calls the other binaries it expects to be available, and if
> > > > > wrapping it would be enough or if we would need to just propagate the
> > > > > other packages for functionality.
> > > >
> > > > Gitolite simply expects tools like git to be on $PATH.  It's a pretty
> > > > naive system, there's nothing like a configure script that is
> > > > determining the absolute file name of these tools and substituting
> > > > those names into the built files.
> > > >
> > > > The executable is already wrapped so that coreutils, findutils, and
> > > > git are on $PATH, but notably not openssh:
> > > >
> > > > (add-after 'install 'wrap-scripts
> > > >                     (lambda* (#:key inputs outputs #:allow-other-keys)
> > > >                       (let ((out (assoc-ref outputs "out"))
> > > >                             (coreutils (assoc-ref inputs "coreutils"))
> > > >                             (findutils (assoc-ref inputs "findutils"))
> > > >                             (git (assoc-ref inputs "git")))
> > > >                         (wrap-program (string-append out "/bin/gitolite")
> > > >                           `("PATH" ":" prefix
> > > >                             ,(map (lambda (dir)
> > > >                                     (string-append dir "/bin"))
> > > >                                   (list out coreutils findutils git)))))))
> > > >
> > > > However, git and openssh are still propagated inputs. I'm going to
> > > > move the propagated inputs to regular inputs, potentially add openssh
> > > > to the wrapper once I remind myself what gitolite does with those
> > > > tools, and test it all out on my server using the gitolite service.
> > > > If that all works, we have a good starting point for adding extension
> > > > support in the service.
> > >
> > > I like it. Let us know how it goes.
> >
> > The problem is that gitolite generates git hooks for the repositories
> > that it manages, and those hooks invoke git, so the only way those
> > scripts will be able to work (without input propagation) is to find a
> > way to inject the proper PATH or find a way to replace references to
> > things like 'git diff' with '/gnu/store/.../git diff'.  I'm going to
> > keep exploring and report back when I have something to show.
> 
> After several rounds of experimentation and breaking my git server a
> few times, here's what I've found:
> 
> * Changing git and openssh to be regular inputs and wrapping both
> gitolite and gitolite-shell with a $PATH that contains git works and
> it's very little extra code.
> 
> * Trying to replace every invocation of a git command took a lot of
> grepping and crafting of regexps to use for substitute* and I never
> got to a point where the result wasn't buggy.  In particular,
> gitolite-shell never worked properly so I couldn't push to my repos.
> 
> So, I think the simple wrapper approach is the way to go.  Patch
> attached.  I tested on my git server by making changes to my gitolite
> configuration and pushing those changes to the special gitolite-admin
> repo.  This causes gitolite to refresh internal configuration using a
> git hook, so I know that hooks can find the executables they need.
> That plus the 'gitolite setup' invocation made by the service
> activation script covers a fair amount of surface area, so I feel
> comfortable committing it.  What do you think?
> 
> Once this part is done, I'll turn my attention to the optional extensions.

Overall it looks good to me. I was going to ask about inetutils and
openssh since they're not wrapping the binaries but I see their paths
are substituted in the 'patch-source phase.

LGTM!

> From 413f2d28aa8bea2274b74c2b574fb9f8bf9c16ba Mon Sep 17 00:00:00 2001
> From: David Thompson <dthompson2 <at> worcester.edu>
> Date: Fri, 2 Sep 2022 14:33:01 -0400
> Subject: [PATCH] gnu: gitolite: Wrap programs instead of using propagated
>  inputs.
> 
> * gnu/packages/version-control.scm (gitolite)[arguments]: Add git to wrapped
> $PATH and additionally wrap gitolite-shell.
> [inputs]: Add git and openssh.
> [propagated-inputs]: Remove it.
> ---
>  gnu/packages/version-control.scm | 18 ++++++++----------
>  1 file changed, 8 insertions(+), 10 deletions(-)
> 
> diff --git a/gnu/packages/version-control.scm b/gnu/packages/version-control.scm
> index 15a9278fe8..1c775932c0 100644
> --- a/gnu/packages/version-control.scm
> +++ b/gnu/packages/version-control.scm
> @@ -1573,17 +1573,15 @@ (define-public gitolite
>                              (coreutils (assoc-ref inputs "coreutils"))
>                              (findutils (assoc-ref inputs "findutils"))
>                              (git (assoc-ref inputs "git")))
> -                        (wrap-program (string-append out "/bin/gitolite")
> -                          `("PATH" ":" prefix
> -                            ,(map (lambda (dir)
> -                                    (string-append dir "/bin"))
> -                                  (list out coreutils findutils git))))))))))
> +                        (for-each (lambda (file-name)
> +                                    (wrap-program (string-append out file-name)
> +                                      `("PATH" ":" prefix
> +                                        ,(map (lambda (dir)
> +                                                (string-append dir "/bin"))
> +                                              (list out coreutils findutils git)))))
> +                                  '("/bin/gitolite" "/bin/gitolite-shell"))))))))
>      (inputs
> -     (list bash-minimal perl coreutils findutils inetutils))
> -    ;; git and openssh are propagated because trying to patch the source via
> -    ;; regexp matching is too brittle and prone to false positives.
> -    (propagated-inputs
> -     (list git openssh))
> +     (list bash-minimal git perl coreutils findutils inetutils openssh))
>      (home-page "https://gitolite.com")
>      (synopsis "Git access control layer")
>      (description
> -- 
> 2.37.2
> 


-- 
Efraim Flashner   <efraim <at> flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[signature.asc (application/pgp-signature, inline)]

Message #68 received at 25957 <at> debbugs.gnu.org (full text, mbox):

From: "Thompson, David" <dthompson2 <at> worcester.edu>
To: Efraim Flashner <efraim <at> flashner.co.il>, "Thompson,
 David" <dthompson2 <at> worcester.edu>, 
 zimoun <zimon.toutoune <at> gmail.com>, 25957 <at> debbugs.gnu.org
Subject: Re: bug#25957: gitolite broken: created repositories keep references
 to /usr/bin for hooks
Date: Sun, 4 Sep 2022 09:26:28 -0400

Hi Efraim,

On Sun, Sep 4, 2022 at 3:26 AM Efraim Flashner <efraim <at> flashner.co.il> wrote:
>
> Overall it looks good to me. I was going to ask about inetutils and
> openssh since they're not wrapping the binaries but I see their paths
> are substituted in the 'patch-source phase.
>
> LGTM!

Thanks! I made one minor tweak to sort the inputs list alphabetically
and pushed as commit 1aa46a7e29c5bd892219fe20fefb883d2103e29e.

I also pushed a follow-up commit
e4ccfcb22ad96e71ca4dfad95af5aa6229ed9869 that swaps out 'git' for
'git-minimal', saving about 75MiB in the package closure.

I think, technically speaking, this bug has been resolved.  There are
no longer /usr/bin, /usr/sbin, etc. references in our gitolite
package, so extensions should work as long as the user adds the
relevant packages to their user or system profile.  I will keep this
bug open for the moment, though, since I haven't gotten to the final
patch I said I would submit which will make those optional
dependencies easy to add via the gitolite service.  Stay tuned!

- Dave

Message #71 received at 25957 <at> debbugs.gnu.org (full text, mbox):

From: zimoun <zimon.toutoune <at> gmail.com>
To: "Thompson, David" <dthompson2 <at> worcester.edu>, Efraim Flashner
 <efraim <at> flashner.co.il>, "Thompson,
 David" <dthompson2 <at> worcester.edu>, 25957 <at> debbugs.gnu.org
Subject: Re: bug#25957: gitolite broken: created repositories keep
 references to /usr/bin for hooks
Date: Mon, 05 Sep 2022 10:16:06 +0200

Hi,

On dim., 04 sept. 2022 at 09:26, "Thompson, David" <dthompson2 <at> worcester.edu> wrote:

> Thanks! I made one minor tweak to sort the inputs list alphabetically
> and pushed as commit 1aa46a7e29c5bd892219fe20fefb883d2103e29e.

Cool!

> I also pushed a follow-up commit
> e4ccfcb22ad96e71ca4dfad95af5aa6229ed9869 that swaps out 'git' for
> 'git-minimal', saving about 75MiB in the package closure.

Neat!

> I think, technically speaking, this bug has been resolved.  There are
> no longer /usr/bin, /usr/sbin, etc. references in our gitolite
> package, so extensions should work as long as the user adds the
> relevant packages to their user or system profile.  I will keep this
> bug open for the moment, though, since I haven't gotten to the final
> patch I said I would submit which will make those optional
> dependencies easy to add via the gitolite service.  Stay tuned!

Ok, thanks for almost closing this old bugs. :-)


Cheers,
simon

Message #74 received at 25957 <at> debbugs.gnu.org (full text, mbox):

From: "Thompson, David" <dthompson2 <at> worcester.edu>
To: zimoun <zimon.toutoune <at> gmail.com>
Cc: Efraim Flashner <efraim <at> flashner.co.il>, 25957 <at> debbugs.gnu.org
Subject: Re: bug#25957: gitolite broken: created repositories keep references
 to /usr/bin for hooks
Date: Tue, 6 Sep 2022 09:50:24 -0400

On Mon, Sep 5, 2022 at 5:33 AM zimoun <zimon.toutoune <at> gmail.com> wrote:
>
> > I also pushed a follow-up commit
> > e4ccfcb22ad96e71ca4dfad95af5aa6229ed9869 that swaps out 'git' for
> > 'git-minimal', saving about 75MiB in the package closure.
>
> Neat!

Unfortunately, it was so neat that it broke the system test for the
gitolite service so I had to revert it.

- Dave

Message #77 received at 25957 <at> debbugs.gnu.org (full text, mbox):

From: "Thompson, David" <dthompson2 <at> worcester.edu>
To: zimoun <zimon.toutoune <at> gmail.com>
Cc: Efraim Flashner <efraim <at> flashner.co.il>, 25957 <at> debbugs.gnu.org
Subject: Re: bug#25957: gitolite broken: created repositories keep references
 to /usr/bin for hooks
Date: Thu, 6 Oct 2022 09:26:38 -0400

[Message part 1 (text/plain, inline)]

Hello again Simon and Efraim,

On Mon, Sep 5, 2022 at 5:33 AM zimoun <zimon.toutoune <at> gmail.com> wrote:
>
> Hi,
>
> On dim., 04 sept. 2022 at 09:26, "Thompson, David" <dthompson2 <at> worcester.edu> wrote:
>
> > Thanks! I made one minor tweak to sort the inputs list alphabetically
> > and pushed as commit 1aa46a7e29c5bd892219fe20fefb883d2103e29e.
>
> Cool!
>
> > I also pushed a follow-up commit
> > e4ccfcb22ad96e71ca4dfad95af5aa6229ed9869 that swaps out 'git' for
> > 'git-minimal', saving about 75MiB in the package closure.
>
> Neat!
>
> > I think, technically speaking, this bug has been resolved.  There are
> > no longer /usr/bin, /usr/sbin, etc. references in our gitolite
> > package, so extensions should work as long as the user adds the
> > relevant packages to their user or system profile.  I will keep this
> > bug open for the moment, though, since I haven't gotten to the final
> > patch I said I would submit which will make those optional
> > dependencies easy to add via the gitolite service.  Stay tuned!
>
> Ok, thanks for almost closing this old bugs. :-)

Some news: I have updated the gitolite package to use G-expressions.
The package builds and the gitolite system test passes so I pushed
that change to master a little while ago.  That patch has made the
(hopefully) final step in this saga easier. The attached patch
introduces a 'make-gitolite' procedure that can be used to add
arbitrary packages to the wrappers for the gitolite and gitolite-shell
programs.  The return value of this procedure can be used in the
gitolite service configuration to enable the desired optional features
like Redis or git-annex.  The base package inputs are unchanged and
the gitolite system test still passes.

What do you think?

- Dave

[0001-gnu-version-control-Add-make-gitolite-procedure.patch (text/x-patch, attachment)]

Message #80 received at 25957 <at> debbugs.gnu.org (full text, mbox):

From: "Thompson, David" <dthompson2 <at> worcester.edu>
To: zimoun <zimon.toutoune <at> gmail.com>
Cc: Efraim Flashner <efraim <at> flashner.co.il>, 25957 <at> debbugs.gnu.org
Subject: Re: bug#25957: gitolite broken: created repositories keep references
 to /usr/bin for hooks
Date: Thu, 6 Oct 2022 09:42:16 -0400

I made a small mistake in the last patch and forgot to include '(gnu
packages version-control)' in the 'use-modules' form in the example
within the manual.  I have fixed that in my local copy.

- Dave

Message #83 received at 25957 <at> debbugs.gnu.org (full text, mbox):

From: zimoun <zimon.toutoune <at> gmail.com>
To: "Thompson, David" <dthompson2 <at> worcester.edu>
Cc: Efraim Flashner <efraim <at> flashner.co.il>, 25957 <at> debbugs.gnu.org
Subject: Re: bug#25957: gitolite broken: created repositories keep
 references to /usr/bin for hooks
Date: Sat, 08 Oct 2022 16:56:08 +0200

Hi Dave,

On Thu, 06 Oct 2022 at 09:26, "Thompson, David" <dthompson2 <at> worcester.edu> wrote:

> Some news: I have updated the gitolite package to use G-expressions.
> The package builds and the gitolite system test passes so I pushed
> that change to master a little while ago.  That patch has made the
> (hopefully) final step in this saga easier. The attached patch
> introduces a 'make-gitolite' procedure that can be used to add
> arbitrary packages to the wrappers for the gitolite and gitolite-shell
> programs.  The return value of this procedure can be used in the
> gitolite service configuration to enable the desired optional features
> like Redis or git-annex.  The base package inputs are unchanged and
> the gitolite system test still passes.

Thank you for working on this!  Neat.

Well, your proposal LGTM although I do not see the difference between
’make-gitolite’ and a simple ’inherent’ for building a package variant.
Yeah, this make-gitolite is probably more handy.

I do not have a strong opinion on the matter. :-)

Cheers,
simon

Message #86 received at 25957 <at> debbugs.gnu.org (full text, mbox):

From: "Thompson, David" <dthompson2 <at> worcester.edu>
To: zimoun <zimon.toutoune <at> gmail.com>
Cc: Efraim Flashner <efraim <at> flashner.co.il>, 25957 <at> debbugs.gnu.org
Subject: Re: bug#25957: gitolite broken: created repositories keep references
 to /usr/bin for hooks
Date: Sat, 8 Oct 2022 21:40:09 -0400

On Sat, Oct 8, 2022 at 11:16 AM zimoun <zimon.toutoune <at> gmail.com> wrote:
>
> Hi Dave,
>
> On Thu, 06 Oct 2022 at 09:26, "Thompson, David" <dthompson2 <at> worcester.edu> wrote:
>
> > Some news: I have updated the gitolite package to use G-expressions.
> > The package builds and the gitolite system test passes so I pushed
> > that change to master a little while ago.  That patch has made the
> > (hopefully) final step in this saga easier. The attached patch
> > introduces a 'make-gitolite' procedure that can be used to add
> > arbitrary packages to the wrappers for the gitolite and gitolite-shell
> > programs.  The return value of this procedure can be used in the
> > gitolite service configuration to enable the desired optional features
> > like Redis or git-annex.  The base package inputs are unchanged and
> > the gitolite system test still passes.
>
> Thank you for working on this!  Neat.
>
> Well, your proposal LGTM although I do not see the difference between
> ’make-gitolite’ and a simple ’inherent’ for building a package variant.
> Yeah, this make-gitolite is probably more handy.

The reason for the constructor is so that extra packages can be easily
added to the gexp that calls wrap-program.  It would be much harder to
modify the package in this way without a helper procedure.

- Dave

Message #89 received at 25957 <at> debbugs.gnu.org (full text, mbox):

From: "Thompson, David" <dthompson2 <at> worcester.edu>
To: zimoun <zimon.toutoune <at> gmail.com>
Cc: 25957-done <at> debbugs.gnu.org, Efraim Flashner <efraim <at> flashner.co.il>,
 25957 <at> debbugs.gnu.org
Subject: Re: bug#25957: gitolite broken: created repositories keep references
 to /usr/bin for hooks
Date: Mon, 24 Oct 2022 17:21:44 -0400

On Sat, Oct 8, 2022 at 9:40 PM Thompson, David <dthompson2 <at> worcester.edu> wrote:
>
> On Sat, Oct 8, 2022 at 11:16 AM zimoun <zimon.toutoune <at> gmail.com> wrote:
> >
> > Hi Dave,
> >
> > On Thu, 06 Oct 2022 at 09:26, "Thompson, David" <dthompson2 <at> worcester.edu> wrote:
> >
> > > Some news: I have updated the gitolite package to use G-expressions.
> > > The package builds and the gitolite system test passes so I pushed
> > > that change to master a little while ago.  That patch has made the
> > > (hopefully) final step in this saga easier. The attached patch
> > > introduces a 'make-gitolite' procedure that can be used to add
> > > arbitrary packages to the wrappers for the gitolite and gitolite-shell
> > > programs.  The return value of this procedure can be used in the
> > > gitolite service configuration to enable the desired optional features
> > > like Redis or git-annex.  The base package inputs are unchanged and
> > > the gitolite system test still passes.
> >
> > Thank you for working on this!  Neat.
> >
> > Well, your proposal LGTM although I do not see the difference between
> > ’make-gitolite’ and a simple ’inherent’ for building a package variant.
> > Yeah, this make-gitolite is probably more handy.
>
> The reason for the constructor is so that extra packages can be easily
> added to the gexp that calls wrap-program.  It would be much harder to
> modify the package in this way without a helper procedure.

Pushed as commit 966118da711506b04c11fbfcac9483d59ed2d912.  This bug
can finally be closed!

- Dave

Message #97 received at 25957 <at> debbugs.gnu.org (full text, mbox):

From: zimoun <zimon.toutoune <at> gmail.com>
To: "Thompson, David" <dthompson2 <at> worcester.edu>
Cc: 25957-done <at> debbugs.gnu.org, Efraim Flashner <efraim <at> flashner.co.il>,
 25957 <at> debbugs.gnu.org
Subject: Re: bug#25957: gitolite broken: created repositories keep
 references to /usr/bin for hooks
Date: Tue, 25 Oct 2022 11:58:06 +0200

Hi Dave,

On Mon, 24 Oct 2022 at 17:21, "Thompson, David" <dthompson2 <at> worcester.edu> wrote:

> Pushed as commit 966118da711506b04c11fbfcac9483d59ed2d912.  This bug
> can finally be closed!

\o/ Awesome!  Thank you!

Cheers,
simon

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.