Let flutter be installable via homebrew

[xster]

Latest update: #14050 (comment)

A description of the desired user experience: #14050 (comment)

---

Didn't find any existing issues on this. Opening for tracking.

Let flutter be brew install flutterable.

[xster]

@gspencergoog you might be interested

[xster]

https://docs.brew.sh/Formula-Cookbook

The process is pretty straight-forward. Though we'd have to make sure it integrates with our https://github.com/flutter/flutter/wiki/Release-process, ideally in an automated way.

[briankung]

Hi Flutter team! I started adding a flutter formula to Homebrew at Homebrew/homebrew-core#27395, but I could definitely use more guidance on following the direction in this comment:

Though we'd have to make sure it integrates with our https://github.com/flutter/flutter/wiki/Release-process, ideally in an automated way.

Right now the Homebrew formula just downloads the current beta release, unzips it, and copies it to Flutter's Keg (eg. /usr/local/Cellar/flutter/0.3.1/).

Any feedback would be welcome!

[xster]

@jcollins-g wrt to the shlock from Homebrew/homebrew-core#27395, maybe we can check if shlock returns not permitted / 1 and parent process id is brew, don't shlock. Also we can lock write to parts of the cache to prevent self update when brew installed.

[briankung]

For context in that Homebrew PR, I wrote about the shlock issue in the comments starting here: https://github.com/Homebrew/homebrew-core/pull/27395/files#diff-b438c13e924f5f36b1ce4398a89436d2R36

[xster]

Actually, we went through this idea a bit more and I think ultimately, we'd want the official formula to be in a homebrew tap (non-core repository). This way, we can keep the flutter upgrade and brew upgrade targets hermetically synchronized.

Hopefully we can bring a version of your formula into the tap in the near future and simplify the official install process.

[briankung]

Sure! In that case, I'd be happy to close the PR on homebrew-core if that makes sense?

[xster]

Yes. Thanks for making the contribution!

[ilovezfs]

@xster fwiw that's not necessary. Formulae in Homebrew/homebew-core get upgraded within hours of upstream releases since we run brew livecheck several times per day.

[jcollins-g]

@xster / @briankung shlock's behavior in brew is a little surprising to me. It's not like much special is going on here:

https://opensource.apple.com/source/shell_cmds/shell_cmds-118/shlock/shlock.c.auto.html

openloop:
	if ((fd = open(tempname, O_RDWR|O_CREAT|O_EXCL, 0644)) < 0) {

That should always be valid to do on a POSIX filesystem -- well, unless you've set chflags on the directory to prevent it from being written:

jcollins-macbookpro:shlock jcollins$ sudo chflags uchg .
jcollins-macbookpro:shlock jcollins$ ls -la
total 0
drwxr-xr-x   2 jcollins  wheel    64 May  3 08:50 .
drwxrwxrwt  69 root      wheel  2208 May  3 08:52 ..
jcollins-macbookpro:shlock jcollins$ echo hi > foo
-bash: foo: Operation not permitted
jcollins-macbookpro:shlock jcollins$ ls -la
total 0
drwxr-xr-x   2 jcollins  wheel    64 May  3 08:50 .
drwxrwxrwt  69 root      wheel  2208 May  3 08:52 ..
jcollins-macbookpro:shlock jcollins$ shlock -f foo -p $$
shlock: open(shlock14741): Operation not permitted
jcollins-macbookpro:shlock jcollins$

If something in homebrew or the way you're setting it up does that, fixing shlock isn't going to cut it because you'll then fail when other write operations are inevitably attempted in the cache.

If adding this line:

chflags -R nouchg "$FLUTTER_ROOT/bin/cache"

here fixes it, you've got a chflags problem and probably should deal with that somehow. Somehow, bin/cache may already exist at the point you first run flutter (maybe you're starting with a binary build of flutter? Those have cache-premade and so they'll pass the mkdir fine, but you won't be able to write to the cache).

[edited to fix chflags]

[briankung]

@jcollins-g Ah, my unix-fu isn't up to snuff - I didn't know about chflags, but it makes sense. I suppose it's academic at this point, but I'll look into it. Thanks!

[briankung]

I added the line as follows:

diff --git a/bin/flutter b/bin/flutter
index 079c3f94d..1b7e06f43 100755
--- a/bin/flutter
+++ b/bin/flutter
@@ -39,6 +39,7 @@ function _rmlock () {
 
 function upgrade_flutter () {
   mkdir -p "$FLUTTER_ROOT/bin/cache"
+  chflags -R nouchg "$FLUTTER_ROOT/bin/cache"
 
   # This function is executed with a redirect that pipes the source of
   # this script into file descriptor 3.

And the test flutter --version still fails with the shlock error.

I missed something you said earlier:

maybe you're starting with a binary build of flutter? Those have cache-premade and so they'll pass the mkdir fine, but you won't be able to write to the cache

I think I am using a binary build of flutter. It's the zip from https://flutter.io/setup-macos/#get-sdk.

Since I was curious, I deleted the cache folder to see if the flutter script could create the folder itself. It could not: mkdir: /usr/local/Cellar/flutter/0.3.1-beta/bin/cache: Operation not permitted and inserting an echo $(whoami) into flutter shows that the script is running as my personal user (sanity check).

I'm not too familiar with Unix permissions, but it looks like I do need sudo to edit any files in /usr/local/Cellar/flutter/, and reviewing the permissions shows that all the files are in the admin group:

$ pwd && echo && ls -al bin
/usr/local/Cellar/flutter/0.3.1-beta

total 32
drwxr-xr-x   5 briankung  admin   160 May  4 10:01 .
drwxr-xr-x  26 briankung  admin   832 May  4 09:53 ..
-r-xr-xr-x   1 briankung  admin  6892 May  4 10:01 flutter
-r--r--r--   1 briankung  admin  6338 May  4 09:52 flutter.bat
drwxr-xr-x   8 briankung  admin   256 May  4 09:52 internal

Copying the directory into my home directory assigns it the staff group, and then running ~/flutter/bin/flutter --version with the cache directory deleted starts downloading the Dart language. However, changing the group on Homebrew's flutter folder, /usr/local/Cellar/flutter/, does not allow the brew keg to create the folder, and when I create the folder myself, I end up right back at the shlock: open(/usr/local/Cellar/flutter/0.3.1-beta/bin/cache/shlock9849): Operation not permitted error.

[jcollins-g]

Thanks for the detailed response @briankung. I could dive further into this with you, but it sounds like the fundamental bug is that we can't guarantee writeability into bin/cache under the flutter installation by regular users and Flutter assumes this.

Most Unix packages separate the cached bits they manage from the binary installation. Flutter's packaging method is a bit unorthodox in that the cached bits include most of the binary installation. Homebrew, apt-get, virtually all package managers and operating systems will have some combination of permissions, chroot during build, chattr/chflags, etc to try to prevent installed packages from being modified by regular users.

I think the right thing to do here might be to have a flag set in a file that locks the flutter cache as read-only, and make flutter's built-in update tools respect it. That way we can distinguish between user-configuration error and the user using an installed package, and display appropriate errors. Or, more simply, we could just check for cache writeability and skip it if we can't write to the cache.

[jcollins-g]

@xster's original suggestion might be enough if the rest of the tools already support this. which sounds like it might be the case.

[jcollins-g]

I've done a little more digging. It looks like we mostly do not need to write to FLUTTER_ROOT and its subdirectories.

However, there are exceptions that look likely to result in crashes:

The shlock we're running into here is definitely one.
The locking in flutter upgrade is another.
The locking for golden files is another.

Additionally, the preference for a flutter root pub cache looks suspicious to me -- unless pub is coded to handle this case that's probably not going to work for cases where a developer wants to use a different package, either.

Given that I found this many little issues right away, I think to fix this for good we probably need to add running integration tests using a read-only FLUTTER_ROOT and track down any little one-offs like this. That'll also keep this from cropping up again as it is really easy as a Flutter developer to accidentally introduce something like this.

[xster]

+1 on testing with read-only root

[gspencergoog]

@CAoTx Flutter's site doesn't have any ads on it, and I have no problems downloading with ad blockers installed. Perhaps your custom adblock filters are too restrictive? The site itself is open source, if you want to see how to adjust your filters.

Another alternative is to use git clone as described on the bottom of the downloads page for getting the master channel. You can use that, and then use flutter channel to switch to another channel if you don't want to use master.

[lukepighetti]

Onboarding new macOS user to flutter development and the installation process is archaic and confusing at best. brew install flutter should be doable and supported by the core team as the official installation method for flutter.

[ikurek]

Since this issue has been labeled with P6, which is the lowest priority possible, I guess it won't be solved anytime soon? It's pretty funny, since this is one of the most commented and active issues in repository.

[lukepighetti]

@timsneath is there any way we can get more transparency into how these community requests are prioritized?

[eseidelGoogle]

https://github.com/flutter/flutter/wiki/Triage
https://github.com/flutter/flutter/wiki/Issue-hygiene
talk about the processes.

Although our teams do not currently have bandwidth to take on this issue, that doesn't mean someone else can't. This would be a medium/large endeavor, but should be doable by a motivated contributor.

There are a bunch of questions I could imagine need answering here:

• Teaching flutter_tools (and other scripts) not to write into the sdk install directory but rather use other cache directories (this may be useful for code-hygiene regardless).
• Teaching fluttter_tools that it's OK to have an un-upgradable (non-writable) install and to somehow point users to different upgrade methods.
• Is Flutter's git-based installation method compatible with these tools? We already offer tarballs/zips for initial download, but currently we assume users install flutter/flutter as a git repo. If it's not compatible, there may be some non-trivial process work on the infrastructure side to support producing/distributing/testing non-git distributions of flutter.

Happy to argue about priorities (maybe in another forum), but I'd encourage looking through what other bugs are labeled higher priority than this one. Flutter has a lot of users and from those a lot of (sometimes urgent) requests. We try to address them in the order of making the most impact towards our/users goals.

[lukepighetti]

@eseidelGoogle Thanks for the clarification, and the strategy sketch, it is very much appreciated and super empowering!

[gleich]

To answer the question about updating I think if the regular flutter update command can just not be available for not git installs then that should be good enough. Is there a way to do that? Ideally, it wouldn't be another distribution, just an if check in the code to see if the location of the flutter binary has a .git folder. Is this possible?

[gleich]

The formula could also just download the git repository and run the commands for a user. Formulas don't have to just install binaries.

[hronro]

As a reference, this is how rustup does:

rustup, the toolchain manager of the Rust programming language, it can be installed by running a installing script (standalone installation) or installed from a OS package manager. When you want to upgrade rustup, you can run rustup self update for standalone installation, or upgrade via your OS package manager if you installed rustup via OS package manager. If you try to run rustup self update when rustup is installed via OS package manager, it will give you an error, says please upgrade it from your OS package manager. I really like this implementation, because when you firstly installed something via your OS package manager and upgrade it using the built-in functionality of it, it may run out of control of the OS package manager and may cause some unknown errors in the future.

I think the best way for flutter is to adopt similar implementation as rustup does, but this is a huge work (need to refactor many things to let flutter can managed by a system package manager) and may requires the help from core team members of flutter, so I just give my suggestions here and I won't expect this can be done soon.

[lukepighetti]

Yeah it sounds waaaaay more complicated than the alleged 10 liner that was referenced earlier in this thread.

[eseidelGoogle]

TIL there are more discussion on this going on: http://flutter.dev/go/sdk-per-platform-installers

[listepo]

Homebrew Formulae https://formulae.brew.sh/cask/flutter

[maxbritto]

Homebrew Formulae https://formulae.brew.sh/cask/flutter

I've used this homebrew cask to install Flutter on a fresh BigSur install and it has worked flawlessly so far ; it has been a month.
Flutter has even been able to auto-upgrade itself to version 2.0 using flutter upgrade from an initial install on 1.22.
I haven't tried switching to a beta release but for regular use it seem to be working for me

[nerdgore]

There is an unofficial formulae that always installs the latest version. https://github.com/flschweiger/homebrew-flutter

[Harmelodic]

So... are we done?

[Hixie]

Latest status here is that nobody working on Flutter's release processes currently has expressed an interest in implementing this. We don't really see this as a high priority compared to our other release-related work (such as working towards SLSA compliance) since there are a number of other mechanisms for obtaining Flutter today. That said, we recognize that homebrew is a pretty idiomatic way of getting software for developers on macOS, and so the request is quite valid.

If anyone would be interested in implementing an official homebrew installation path, the best thing to do would be to reach out on the #hackers-releases channel of our Discord (see the contributing docs for details). Implementing it would require integrating into our release pipeline, so familiarity with that would be extremely helpful. It would also require carefully negotiating how Flutter's primary distribution mechanism (shipping the git repo directly) should interact with Homebrew's mechanisms, so familiarity with both of those would also be needed.

[fzyzcjy]

By the way, https://github.com/fluttertools/fvm does support install via brew: brew tap leoafarias/fvm; brew install fvm. Then use fvm to install any version of flutter (and manage multi flutters which is the primary usage).