Trying to understand why /home/linuxbrew must be used.

[anderspitman]

Additional Info

• I am using Windows Subsystem for Linux

Output of brew config

Not using brew yet

Output of brew doctor

Not using brew yet

Description of issue

I'm trying to find a package manager specifically for installing software on systems where I don't have root.

I've read through a few related issues, but I still don't quite understand why it's necessary to build from source if you don't use /home/linuxbrew. Most executables I've downloaded can be run from anywhere. Is it specifically because homebrew sometimes uses its own libc et al, and there are some hard-coded rpaths? In that case would it be possible to patch the rpaths as exodus does?

[SMillerDev]

I've read through a few related issues, but I still don't quite understand why it's necessary to build from source if you don't use /home/linuxbrew.

It isn't always needed. Only if brew determined it couldn't use the prebuilt binary.

Most executables I've downloaded can be run from anywhere.

Then you got lucky

Is it specifically because homebrew sometimes uses its own libc et al, and there are some hard-coded rpaths?

It always uses it's own dependencies and sometimes locations are hardcoded into a build which will cause issues.

In that case would it be possible to patch the rpaths as exodus does?

It's not just RPATH as far as I know, but if you can patch all the locations I'm sure others would appreciate if you made a PR to implement this.

[anderspitman]

Then you got lucky

Maybe we're talking about different things. I maybe be using the wrong terminology. On my machine if I run

cp /usr/bin/grep .
./grep

it runs fine. Now if I were to move it to a different machine (perhaps with an old glibc) I would not necessarily expect it to work. But the filesystem path does not affect the program's ability to run on a given machine.

It's not just RPATH as far as I know

Do you have any examples of what else it might be?

[SMillerDev]

it runs fine

Grep is not a great test case here since it depends on almost nothing. Try something like httpd or ffmpeg.

Do you have any examples of what else it might be?

Any file involved in the build could be referenced in the resulting binary. If you're lucky it's just debug data. If you're unlucky it hardcodes where tools are on the filesystem.

[anderspitman]

I just tried it with ffmpeg, which on my system has 167 linked dependencies (thanks for the much more realistic example). It works perfectly. I also ran it through exodus, which produced a ~200MB directory with ffmpeg and all its dependencies. That version runs fine in an Alpine docker image, which uses musl and doesn't have glibc.

Do you perhaps have a concrete example of a homebrew package that simply can't work in this way? I know there must be something I'm missing here.

[SMillerDev]

See the other thread

[carlocab]

If you're not bothered by references to the build prefix inside binary files, you could just do something like

brew install --force-bottle

to ignore brew's checks for whether a binary package is relocatable and install it anyway.

Anything you install this way is likely to pass a quick smoke test. I wouldn't rely on them for anything important, though.

[carlocab]

Is it specifically because homebrew sometimes uses its own libc et al, and there are some hard-coded rpaths?

A lot of formulae build with hard-coded paths (i.e. not just rpaths) that encode the prefix registered at build-time, so brew scans binary files for these references. If it finds them then the software must be built from source when your prefix is not /home/.linuxbrew. If it doesn't, brew will let you install the bottle regardless of your prefix.

[carlocab]

Is there a reason that formulae do this?

I would assume so, but that's hard to answer for all of the ~900 or so formulae that do it.

What is the advantage as opposed to my grep example above?

You'll have to ask the upstream developers of the relevant formulae about the advantage, since the hard-coded paths in those formulae are a consequence of upstream design choices.

[anderspitman]

Do you have an example of such a package by chance? Since I'm new to homebrew I have run into this problem in the wild.

[carlocab]

Try this in the Formula directory of a clone of the Homebrew/homebrew-core repository:

grep -E -r 'sha256 \s*(, )?x86_64_linux'

These will be the packages that, after being installed, contain references to the build prefix (i.e. /home/.linuxbrew/*) inside binary files.

I also just remembered that these won't be rpaths, since brew knows how to relocate those.

[carlocab]

I guess one example is watchman, which hardcodes the path to its state directory (and must be an absolute path determined at build-time), which we put inside $HOMEBREW_PREFIX.

[anderspitman]

watchman is a great example. The official instructions tell you to manually create certain directories. Inspecting the executable with objdump -x watchman | grep NEEDED shows

  NEEDED               /usr/local/lib/libglog.so.0
  NEEDED               libcrypto.so.1.1
  NEEDED               /usr/local/lib/libgflags.so.2.2
  NEEDED               libpthread.so.0
  NEEDED               /usr/local/lib/libevent-2.1.so.7
  NEEDED               /usr/local/lib/libsnappy.so.1
  NEEDED               libdl.so.2
  NEEDED               libstdc++.so.6
  NEEDED               libm.so.6
  NEEDED               libgcc_s.so.1
  NEEDED               libc.so.6

So several libraries are hard coded. This can be modified with patchelf, but it still fails because /usr/local/var/run/watchman is hard coded in the program somewhere.

So I agree this is an example of a program that is not trivially relocatable.

However, this is the first time I've run into this sort of thing. It seems like the vast majority of programs wouldn't have such issues.

Also, do you have any idea why watchman does this? Is there some reason it can't use the working directory or XDG directories for its runtime state?