How to set LD_LIBRARY_PATH from within ruby so that FFI picks it up? #938

jonesmz · 2022-02-01T19:16:53Z

I have an in-house library that does not have RPATH set, which is bundled with other in-house libraries, and all of these libraries are deployed to a non-standard folder.

To load these libraries, we expect to set the LD_LIBRARY_PATH to inform the operating system of where to search for dynamic libraries.

On Linux, if I set my LD_LIBRARY_PATH before calling ruby

LD_LIBRARY_PATH=mySpecialPath ruby blahblahblah

Then FFI is able find the library that I want it to, as well as all of it's dependencies.

However, if I set the LD_LIBRARY_PATH inside of ruby, before calling FFI related functionality

ENV['LD_LIBRARY_PATH'] = "mySpecialPath"

Then FFI (or Ruby, whoever is responsible) does not communicate the path to the OS's library loader

I've used the environment variable LD_DEBUG=libs to inspect the search path that is used, and have confirmed that if I use LD_LIBRARY_PATH before starting ruby, the path is present, and if i instead set it during execution of the library, then the path is absent.

Is it possible to set this path inside of ruby?

Or must it be set prior to executing ruby?

headius · 2022-02-03T14:54:02Z

Could you put together a small example? I tried to search for information on this and ended up back here at this issue. My guess is that because Ruby itself loads a number of dynamic libraries at startup, the value of LD_LIBRARY_PATH has already been read by the dynamic linker, and so changing it from Ruby code at runtime will have no effect.

Maybe try full path?

jonesmz · 2022-02-05T21:59:07Z

We have a folder of pre-built libraries.

E.g. path/to/linux/libs/

They're named with a pattern like

mylib-l64r-21-3.so

The naming pattern is specific to my employer, and intentionally does not follow the typical linux shared library naming convention, to avoid accidentally picking up a system library of the same name.

We load a specific library from that folder that depends on the other libs in that same folder by filename, but not full path name, because the directory is not the same on all machines, as it's part of our developer workstation configuration, each dev can pick their own path.

If we add RPATH of "." to the libraries, they find each other fine. This is undesirable, though, since it would require a bunch of annoying changes to our build system to get it to work properly.

If we launch ruby with LD_LIBRARY_PATH=path/to/linux/libs/

then it works fine, though doing that involved playing whack-a-mole with finding all the scripts that might use those libs...

The reason I opened this SCR is that the FFI library's documentation made it seem like it was possible to set the LD_LIBRARY_PATH from within ruby, which would have been perfect, since we calculate where these paths are inside of ruby.

Because it wasn't possible, i needed to convert the "where's the library folder?" logic to bash, and calculate it ahead-of-time in our wrapper script that launches ruby.

If it's not possible to set this during process runtime, that's understandable, but I recommend making a clear note of this platform restriction in the docs so that people like myself don't go down a rabbit hole of trying to figure out why they can't change the LD_LIBRARY_PATH.

To be clear, i'm not saying that FFI should support this. Simply that I thought it might, but I couldn't figure out how.

Part of my confusion was that I saw some comments in another community (it might have been ruby on rails, maybe??) saying that the trick was to make sure the LD_LIBRARY_PATH was set in the "main" runner, and not any "children". Since I am a complete Ruby noob, and inherited this code from someone who left the company 10 years ago, and no one's touched it since, i had no idea whether that advice was applicable, but it made me think there might have been some mechanism inside of the FFI library that could be exposed to make this a first-class capability.

brauliobo · 2024-09-26T22:17:45Z

@jonesmz got the same here. it works setting LD_LIBRARY_PATH outside of ruby but not inside ruby with ENV['LD_LIBRARY_PATH'] =

did you get a solution?

jonesmz · 2024-11-12T19:59:59Z

@jonesmz got the same here. it works setting LD_LIBRARY_PATH outside of ruby but not inside ruby with ENV['LD_LIBRARY_PATH'] =

did you get a solution?

No, I never solved this.

I worked around the problem by avoiding the FFI usage entirely.

larskanis · 2024-11-12T21:05:14Z

There are several ways to locate and load a library and it's dependencies on Linux with their own advantages and disadvantages. I try to sum them up here:

Add the path of the library(s) to /etc/ld.so.conf.d/* files and call sudo ldconfig to update the cache. Then the library to be used by FFI can be loaded by just the library file name. This obviously requires root permissions and must be done before the FFI running process is executed.
Add the path of the library(s) to LD_LIBRARY_PATH. Then the library to be used by FFI can be loaded by just the library file name. This must be set by the parent process of the FFI running process.
Dependent libraries can be located by the rpath compiled into the so file, which depends on some other so file. It can be a absolute path or a relative path. A pure relative path depends on the current working directory, while a relative path based on $ORIGIN locates dependent libraries relative to the depending library. The letter is usually preferred. This way requires to adjust the linker options at build time (-Wl,-rpath,'$ORIGIN' or so). It can't be set at runtime. See for instance here.
Libraries can be loaded per absolute path into the FFI process. This can be done by ffi_lib(File.expand_path('libexample.so', __dir__)) or similar. If there are dependent .so files they can be loaded in the same process prior to the ffi_lib call by FFI::DynamicLibrary.new("/absolut/path/to/libxyz.so", 0). But the dependent libraries must be loaded in the right order, so that all dependencies are loaded before the depending library. This way libraries without rpath can be loaded independent of environment variables of the parent process, but the resolution of the dependency tree must be done manually then.

The library loading is mostly an issue of the operating system and out of scope of the FFI gem. Therefore we can not do much to support other use cases.

tasaif · 2025-05-07T00:57:53Z

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

How to set LD_LIBRARY_PATH from within ruby so that FFI picks it up? #938

How to set LD_LIBRARY_PATH from within ruby so that FFI picks it up? #938

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

How to set LD_LIBRARY_PATH from within ruby so that FFI picks it up? #938

How to set LD_LIBRARY_PATH from within ruby so that FFI picks it up? #938

Comments

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!