GNU bug report logs - #21076
dynamic-link often fails to load libraries

Previous Next

Package: guile;

Reported by: Frank Webster <f.webster <at> yandex.com>

Date: Thu, 16 Jul 2015 17:05:02 UTC

Severity: normal

Done: Ludovic Courtès <ludo <at> gnu.org>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 21076 in the body.
You can then email your comments to 21076 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-guile <at> gnu.org:
bug#21076; Package guile. (Thu, 16 Jul 2015 17:05:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Frank Webster <f.webster <at> yandex.com>:
New bug report received and forwarded. Copy sent to bug-guile <at> gnu.org. (Thu, 16 Jul 2015 17:05:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Frank Webster <f.webster <at> yandex.com>
To: bug-guile <at> gnu.org
Subject: dynamic-link often fails to load libraries
Date: Thu, 16 Jul 2015 19:52:34 +0300
On my Debian system, `dynamic-link' often fails to load shared
libraries:

Loading using a versioned soname never works:

scheme@(guile-user)> (dynamic-link "libGL.so.1")
ERROR: In procedure dynamic-link:
ERROR: In procedure dynamic-link: file: "libGL.so.1", message: "file not found"

Loading without a library extension doesn't work unless the
corresponding *-dev package is installed:

scheme@(guile-user)> (dynamic-link "libGL")
ERROR: In procedure dynamic-link:
ERROR: In procedure dynamic-link: file: "libGL", message: "file not found"

It is supposed to work with or without the extension, however because
of limitations in the underlying lt_dlopenext function in libtool,
neither works.

The first doesn't work because contrary to its
documentation, lt_dlopenext doesn't always try loading
the bare filename first, without appending an extention.

The second doesn't work because the versioned soname extension (.so.1)
isn't among the ones that lt_dlopenext tries to append. It does try the
unversioned extension (.so), but in Debian the unversioned symlink
is only available when the corresponding *-dev package is installed.

Here is a related libtool bug report: https://debbugs.gnu.org/8976

Note that there are arguably two distinct issues:
(1) lt_dlopenext not trying the bare filename in all cases
(2) lt_dlopenext not trying versioned soname extensions.

Obviously one way to address this would be to fix these two issues in
lt_dlopenext in libtool.

Alternatively, the (system foreign) module in guile could provide a
simple low-level wrapper around lt_dlopen, and possibly implement
the higher level functionality of `dynamic-link' (additional search
paths, guessing extensions etc.) in scheme.




Information forwarded to bug-guile <at> gnu.org:
bug#21076; Package guile. (Thu, 16 Jul 2015 23:01:01 GMT) Full text and rfc822 format available.

Message #8 received at 21076 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Rottmann <a.rottmann <at> gmx.at>
To: Frank Webster <f.webster <at> yandex.com>
Cc: 21076 <at> debbugs.gnu.org
Subject: Re: bug#21076: dynamic-link often fails to load libraries
Date: Fri, 17 Jul 2015 01:00:23 +0200
Frank Webster <f.webster <at> yandex.com> writes:

> On my Debian system, `dynamic-link' often fails to load shared
> libraries:
>
> Loading using a versioned soname never works:
>
> scheme@(guile-user)> (dynamic-link "libGL.so.1")
> ERROR: In procedure dynamic-link:
> ERROR: In procedure dynamic-link: file: "libGL.so.1", message: "file not found"
>
> Loading without a library extension doesn't work unless the
> corresponding *-dev package is installed:
>
> scheme@(guile-user)> (dynamic-link "libGL")
> ERROR: In procedure dynamic-link:
> ERROR: In procedure dynamic-link: file: "libGL", message: "file not found"
>
> It is supposed to work with or without the extension, however because
> of limitations in the underlying lt_dlopenext function in libtool,
> neither works.
>
I'd argue that using the second variant (or using `(dynamic-link
"libGL.so")') is kinda inherently broken in an application loading
general-use (public) shared libraries, as opposed to plugins specific to
an application: you have no idea what ABI the library you just loaded
has. This means you may later use of symbols from that library in a way
that violates the ABI.

Unfortunatly, how the ABI is encoded into the shared library name is
platform-dependent, and this is also (as I read between the lines of the
bug thread referenced by you [0]) one of the reasons why the libtool
developers are not sure how a solution to this problem should look
like.

I think to write an application that is somewhat protected against ABI
changes, you need to have a mapping from ABIs supported by your
application to shared library file names (including the version
extension). You might have only one supported ABI, but the mapping is
still platform-dependent, so you have to either have a run-time or
build/install-time switch choosing the appriopriate shared library name.

Now, to make matters worse, you can't do that using libltdl, as you
noticed:

> The first doesn't work because contrary to its
> documentation, lt_dlopenext doesn't always try loading
> the bare filename first, without appending an extention.
>
Yes, this is a really annoying bug, and is what kept me from (trying to)
port sbank[1] (a gobject-introspection binding) to Guile a few years
back. In gobject-introspection, you get the full shared library file
name, along with a machine-readable ABI description, but you can't open
the shared library using the versioned name in Guile.

[1] https://github.com/rotty/sbank

> The second doesn't work because the versioned soname extension (.so.1)
> isn't among the ones that lt_dlopenext tries to append. It does try the
> unversioned extension (.so), but in Debian the unversioned symlink
> is only available when the corresponding *-dev package is installed.
>
Yeah, but IMHO, this kind of use with a bare shared library name
(without SONAME/ABI information) is already misguided in the first
place, unless you have some other way of knowing the ABI. Debian and the
likes not installing .so files (or .la files) by default only exposes
the inherent potential breakage.

[0]
> Here is a related libtool bug report: https://debbugs.gnu.org/8976
>
> Note that there are arguably two distinct issues:
> (1) lt_dlopenext not trying the bare filename in all cases
> (2) lt_dlopenext not trying versioned soname extensions.
>
> Obviously one way to address this would be to fix these two issues in
> lt_dlopenext in libtool.
>
As mentioned, solving problem (2) is not a good idea IMO -- the versions
are there for a reason, and you cannot (should not) guess them.

> Alternatively, the (system foreign) module in guile could provide a
> simple low-level wrapper around lt_dlopen, and possibly implement
> the higher level functionality of `dynamic-link' (additional search
> paths, guessing extensions etc.) in scheme.
>
+1

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.xx.vu/>




Information forwarded to bug-guile <at> gnu.org:
bug#21076; Package guile. (Fri, 24 Jun 2016 08:11:01 GMT) Full text and rfc822 format available.

Message #11 received at 21076 <at> debbugs.gnu.org (full text, mbox):

From: Andy Wingo <wingo <at> pobox.com>
To: Andreas Rottmann <a.rottmann <at> gmx.at>, Frank Webster <f.webster <at> yandex.com>
Cc: 21076 <at> debbugs.gnu.org
Subject: Re: bug#21076: dynamic-link often fails to load libraries
Date: Fri, 24 Jun 2016 10:10:02 +0200
On Fri 17 Jul 2015 01:00, Andreas Rottmann <a.rottmann <at> gmx.at> writes:

>> Alternatively, the (system foreign) module in guile could provide a
>> simple low-level wrapper around lt_dlopen, and possibly implement
>> the higher level functionality of `dynamic-link' (additional search
>> paths, guessing extensions etc.) in scheme.
>>
> +1

Either one of yall want to give a shot at this or something like it?  I
think in an ideal world we could lose the ltdl dependency.  I gave a
first crack at it but got mired in C; I never considered implementing it
in Scheme, which seems like a nice idea.

Andy




Information forwarded to bug-guile <at> gnu.org:
bug#21076; Package guile. (Sat, 08 Jun 2019 14:57:01 GMT) Full text and rfc822 format available.

Message #14 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Isaac Jurado <diptongo <at> gmail.com>
To: bug-guile <at> gnu.org
Subject: Re: bug#21076: dynamic-link often fails to load libraries
Date: Sat, 8 Jun 2019 14:29:32 +0200
[Message part 1 (text/plain, inline)]
I've experienced this issue as well, but the fix is very simple
(attaching Git patch).

The fix does NOT break foreign.test but I have no idea, yet, on how to
test the new behaviour in a platform independent or continuous
integration friendly way.

The commit has been applied to the stable-2.2 branch, but it would
also apply cleanly to master.

Any feedback welcomed.

Best regards.

-- 
Isaac Jurado

"The noblest pleasure is the joy of understanding"
Leonardo da Vinci
[0001-Interpret-dynamic-library-name-as-literal-path-first.patch (text/x-patch, attachment)]

Information forwarded to bug-guile <at> gnu.org:
bug#21076; Package guile. (Sat, 08 Jun 2019 14:57:02 GMT) Full text and rfc822 format available.

Message #17 received at 21076 <at> debbugs.gnu.org (full text, mbox):

From: Isaac Jurado <diptongo <at> gmail.com>
To: 21076 <at> debbugs.gnu.org
Subject: Re: bug#21076: dynamic-link often fails to load libraries
Date: Sat, 8 Jun 2019 14:37:23 +0200
[Message part 1 (text/plain, inline)]
I've experienced this issue as well, but the fix is very simple
(attaching Git patch).

The fix does NOT break foreign.test but I have no idea, yet, on how to
test the new behaviour in a platform independent or continuous
integration friendly way.

The commit has been applied to the stable-2.2 branch, but it would
also apply cleanly to master.

Any feedback welcomed.

Best regards.

-- 
Isaac Jurado

"The noblest pleasure is the joy of understanding"
Leonardo da Vinci
[0001-Interpret-dynamic-library-name-as-literal-path-first.patch (text/x-patch, attachment)]

Information forwarded to bug-guile <at> gnu.org:
bug#21076; Package guile. (Fri, 20 Mar 2020 23:37:01 GMT) Full text and rfc822 format available.

Message #20 received at 21076 <at> debbugs.gnu.org (full text, mbox):

From: Matt Wette <matt.wette <at> gmail.com>
To: 21076 <at> debbugs.gnu.org
Subject: dynamic-link
Date: Fri, 20 Mar 2020 16:36:28 -0700
I have encountered this issue with dynamic-link.
I propose a small patch to Guile (works on v3.0.1) to address it.
At least this allows me to load the file if I know the exact name.

This patch fixes issue with libtool and OSes which use
non-standard extensions for shared libraries.  The libltdl
function lt_dlopenext expects the argument to have a standard
extension or no extention.  Arguments reflecting shared object
files with non-standard extensions will fail to load.  For
example, my system has /usr/lib64/libyaml-0.so.2.  w/o this
patch (dynamic-link "libyaml-0.so.2") does not work; with it,
it does.  Go figure.

--- libguile/dynl.c-orig    2020-03-20 05:56:42.101214929 -0700
+++ libguile/dynl.c    2020-03-20 05:57:40.432893330 -0700
@@ -69,7 +69,10 @@
     handle = lt_dlopen (NULL);
   else
     {
-      handle = lt_dlopenext (fname);
+      handle = lt_dlopen (fname);
+
+      if (handle == NULL)
+         handle = lt_dlopenext (fname);

       if (handle == NULL
 #ifdef LT_DIRSEP_CHAR





Reply sent to Ludovic Courtès <ludo <at> gnu.org>:
You have taken responsibility. (Sat, 21 Mar 2020 23:27:02 GMT) Full text and rfc822 format available.

Notification sent to Frank Webster <f.webster <at> yandex.com>:
bug acknowledged by developer. (Sat, 21 Mar 2020 23:27:02 GMT) Full text and rfc822 format available.

Message #25 received at 21076-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Isaac Jurado <diptongo <at> gmail.com>
Cc: 21076-done <at> debbugs.gnu.org, Matt Wette <matt.wette <at> gmail.com>
Subject: Re: bug#21076: dynamic-link often fails to load libraries
Date: Sun, 22 Mar 2020 00:26:44 +0100
[Message part 1 (text/plain, inline)]
Hi Isaac,

Isaac Jurado <diptongo <at> gmail.com> skribis:

> From b337d412b3c10cabe355df07d2295c4d0a560b10 Mon Sep 17 00:00:00 2001
> From: Isaac Jurado <diptongo <at> gmail.com>
> Date: Sat, 8 Jun 2019 14:00:29 +0200
> Subject: [PATCH] Interpret dynamic library name as literal path first.
>
> Fixes <https://bugs.gnu.org/21076>.
>
> * libguile/dynl.c (sysdep_dyn_link): Try plain lt_dlopen first, to
>   interpret fname as a literal path.
> * doc/ref/api-foreign.texi: Update explanation to describe the new
>   behavior.

It’s taken a while (!), but thanks to Matt Wette’s ping, it’s now
applied on ‘master’ and will make it into 3.0.2.

I slightly modified the documentation wording as shown in the patch
below (the GNU convention is to use “file name” for file names and
“path” for search paths).

Like you wrote, it’s hard to test, but I think it’s OK to leave it as is.

Thanks!

Ludo’.

[Message part 2 (text/x-patch, inline)]
diff --git a/doc/ref/api-foreign.texi b/doc/ref/api-foreign.texi
index 94fabf23c..b0d6c249b 100644
--- a/doc/ref/api-foreign.texi
+++ b/doc/ref/api-foreign.texi
@@ -75,14 +75,14 @@ Scheme object suitable for representing the linked object file.
 Otherwise an error is thrown.  How object files are searched is system
 dependent.
 
-Guile first tries to load @var{library} as a full path to a shared
-library file.  If that fails, then it falls back to interpret
+Guile first tries to load @var{library} as the absolute file name of a shared
+library.  If that fails, it then falls back to interpret
 @var{library} as just the name of some shared library that will be
 searched for in the places where shared libraries usually reside, such
 as @file{/usr/lib} and @file{/usr/local/lib}.
 
 @var{library} should not contain an extension such as @code{.so}, unless
-@var{library} represents the full path to the shared library file.  The
+@var{library} represents the absolute file name to the shared library.  The
 correct file name extension for the host operating system is provided
 automatically, according to libltdl's rules (@pxref{Libltdl interface,
 lt_dlopenext, @code{lt_dlopenext}, libtool, Shared Library Support for

bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Sun, 19 Apr 2020 11:24:10 GMT) Full text and rfc822 format available.

This bug report was last modified 3 years and 344 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.