Unable to mount protected Mac paths after upgrade to Docker 3.0.0

[andyvanosdale]

• I have tried with the latest version of my channel (Stable or Edge)
• I have uploaded Diagnostics
• Diagnostics ID: 822E1E1F-1383-4437-8A4D-4922B6AD72B0/20201210165314

Workarounds

Keep v3.0.0

Disable "Use gRPC Fuse for file sharing" under "Experimental Features" in the Docker Preferences.

Downgrade

Downgrade to v2.5.0.1

Expected behavior

I should be able to mount protected Mac paths from ~

Actual behavior

Mounts denied

$ docker run --rm -it -v ${PWD}:/app alpine
docker: Error response from daemon: Mounts denied: approving /Users/me/Downloads: file does not exist.
ERRO[0000] error waiting for container: context canceled

Information

This is a new problem with 3.0.0. This was working before upgrading to Docker 3.0.0. This happens with any protected path:

• ~/Downloads
• ~/Desktop
• ~/Documents

I am able to mount ~ itself.

/Users is listed in File Sharing:

Docker is allowed access to Downloads and Desktop through Security and Privacy:

• macOS Version: 10.15.7 (19H15)

Diagnostic logs

Docker for Mac: 3.0.0

Steps to reproduce the behavior

1. cd ~/Downloads
2. docker run --rm -it -v ${PWD}:/app alpine

[Vardiak]

This doesn't seem to apply only to Downloads... I cannot mount any project 😓

[andyvanosdale]

@Vardiak What paths are you trying to mount? I can't mount any protected Mac paths under /Users, like Documents, Downloads, or Desktop.

[Vardiak]

@andyvanosdale Any project under /Users/Documents. What is a protected Mac path ? Is there a workaround ?

[andyvanosdale]

I'm trying to find a workaround...

[Ben-Atherton]

I've got the same issue, I can't mount anything under /Users/Ben/Documents

I've managed to get it working again (for now) by downgrading to v2.5.0.0 which can be downloaded at https://docs.docker.com/docker-for-mac/release-notes/#docker-desktop-community-2500

[andyvanosdale]

Relevant Apple Support article about folder privacy

[jfreed]

Having the same issue. Disabling "Use gRPC FUSE for file sharing" fixes it.

[andyvanosdale]

If gRPC Fuse is the culprit, I suspect its some other process doing the mounting that Mac isn't able to expose in the UI.

[andyvanosdale]

Hat tip to @jfreed. Can confirm disabling gRPC Fuse is a workaround. I've updated the description with the details.

[stephen-turner]

Thanks for the report. I think this is a side effect of fixing #4975 where people were scared when the OS told them we were viewing Reminders and Downloads. It looks like we need to think again about the solution.

[alfredotranchedone]

I confirm that disabling "Use gRPC FUSE for file sharing" fixes the problem

[adamoki]

Is there any impact on performance when disabling "Use gRPC FUSE for file sharing"? Anybody can give input on that?

[sfcgeorge]

Tried uninstalling and reinstalling 3.0 but that didn't help. Tried manually adding permissions in Security & Privacy but that didn't help either. 93EE1AF6-928B-4DD0-9DA9-6ABAF23380D2/20201210181155

As I believe FUSE is the default and I imagine most people use bind mounts, it might be worth pulling the 3.0 update until this is fixed.

[Vardiak]

I confirm that disabling "Use gRPC FUSE for file sharing" fixes the problem

When disabling gRPC FUSE, I can't run a container for the second time with the same path shared, I have to restart the daemon every time...

[delfuego]

As I believe FUSE is the default and I imagine most people use bind mounts, it might be worth pulling the 3.0 update until this is fixed.

I kind of agree with this — this is a big breakage, and I've had to manually walk a few of our developers through the downgrade back to 2.5.0.0 in the few hours since this hit.

[glebovdev]

Downgrade to v. 2.5.0.1 (https://desktop.docker.com/mac/stable/49550/Docker.dmg) also possible, no issues so far.

[aniket-patel-ef]

I am also facing the same issue.. Thanks to this thread there is a workaround after hours of debugging

[folt]

I faced the same problem. I cannot view any volumes in my project.

[alphonse92]

Same, Disabling "Use gRPC FUSE for file sharing" fixes it.

[rstropoli]

Same here, Disabling "Use gRPC FUSE" fixed the issue.

[pleminh]

Same problem .
Disable gRPC FUSE fixes it .

ERROR: for project-app Cannot start service project-app: Mounts denied: approving

[ConsiderCode]

Can confirm that 3.0.1 fixed this and I did not need to reset to factory defaults. Thank you @andyvanosdale for raising this issue so quickly and suggesting a work around. It saved us a lot of downtime.

[aditilamkhede]

Yes, update fixed the issue for me too(without disabling "Use gRPC FUSE for file sharing").

[jmaroeder]

Count me as one more vote for the ability to share files from inside ~/Library (or at least document in the release notes that this behavior is unsupported).

My argument for this is that ~/Library/Application Support is the default "application data" directory. If I want to mount data that is generated by other applications running on my system, I either have to copy it out of the default location or deal with some symlinks (and I'm not sure if those are supported).

[stephen-turner]

Thanks for the use cases for (subdirectories of) ~/Library. Obviously this was an urgent fix, and we decided that this was the best compromise that we could do quickly. We do intend to return to it and try and find a way to fix this ticket and #4975 simultaneously.

[dee-kryvenko]

By the way, ~/Library is still blocked. We don't see a use case for sharing that, and it will cause performance problems to do so because the docker logfiles are written there.

Thanks for the use cases for (subdirectories of) ~/Library

I don't want to appear ungrateful for your work or something, but I'm very troubled and concerned while trying to wrap my head around the very idea that somebody even had to give you a use case for something like that. I mean, what makes you think that Docker should be in position to be making decisions as to what user can and can't mount to his own containers on his own system? What was the use case to even have this filtering implemented with no ability for the user to have items added and removed from it in the first place? This seems like a huge overreach from the Docker side and it troubles me very much - like what's next going to be limited for the sake of user's own protection, maybe running unauthorized containers from other than Docker Hub sources should be disallowed?

[dee-kryvenko]

To add some weight, I want to point out that the entire DevOps community (or at least these of us who is trying to do CICD in a containerized way, which is pretty much most of us this days), are very affected right now even with 3.0.1 fix. This is so common for portability purposes to move the entire toolchain into the containers and execute them as containers on our local workstations. It's a great deal of portability and it's crucial. And it's so happens that directories that the user would want to mount to his containers in this scenario are all over the place, ~/.aws and ~/.kube to begin with. Most of them happen to be under ~/Library, build caches are a huge part of it - Go included which happens to be in ~/Library/Caches/go-build.

[dee-kryvenko]

And to add some context (and pardon me for the multiple messages, but they are different aspects of a complex issue and to me they are appear to be logically separate), I would like to point out that there are two major use cases for Docker.

Isolation is one of them and it seems to have the most attention. Surely enough the ability to run applications securely in an isolated way is crucial and it had dramatic impact and completely transformed Software Engineering as we know it today.

But Portability is another one that is often remains in the shadow of the first one, yet should not be underestimated as both of them are yin and yang of the containerized world. Ability to distribute and run not just applications but entire ecosystems at our laptops that guarantees the execution environment across the different users and automated systems is a big deal. One could envision the world with no apt, yum, brew, chocolatey and such where all the software basically distributed as containers - and it would be hard to get to that world if portability principal is neglected in favor of total isolation.

It absolutely and ultimately has to be a user choice as to how much security does he need, since there are absolutely different use cases. Like if I run just the app I'm developing - it's one thing, but what if I run something like Terraform or Helm in my containers and I do intentionally want it to have access to parts of my system that in other scenarios might be considered a vulnerability? Docker cannot and should not be allowed to be making such a fundamental assumptions on behalf of the user.

[dlgoodchild]

@dee-kryvenko I think you could be jumping the gun here on all this.

It is an experimental feature afterall. Disable it if you need ~/Library.

There's clearly a technical reason behind not being able to use ~/Library with gRPC FUSE, so maybe they'll simply address that when there's an evident demand for it and maybe when it's not experimental. Remember, they have to prioritise work, and if this isn't a priority then they can leave it "blocked" and address other more pressing issues.
They could just not expose experimental features until they are fully supported. They are afterall contending with different technologies, and a more restrictive OS than most others. Just cos something is blocked for the time being, it doesn't necessarily imply it's some kind of draconian overreach by Docker-for-mac.

Also, this is a bug report for v3.0.0. Maybe open an issue to raise your concerns against v3.0.1.

[tejksat]

@dlgoodchild please correct me if I am wrong but despite being experimental "Use gRPC FUSE" feature is enabled by default for Docker for Mac:

Docker Desktop now uses gRPC-FUSE for file sharing by default.

@stephen-turner for us in JetBrains this issue keeps affecting our users in 3.0.1 on attempts to run tests with coverage within the Docker integration in PyCharm, RubyMine and PhpStorm (f.e. please see this issue) as we bind ~/Library/Caches/JetBrains subdirectories to the container to retrieve the coverage data.

Unfortunately, as "Use gRPC FUSE" feature is enabled by default, our users experience failures in previously working scenarios. This evidently frustrates users and we only could advise them to disable this feature in Docker for Mac as soon as they reach us (until we prepare the fix and what is even more crucial, until they update their IDEs).

Is it possible to include either ~/Library or a subset of it including ~/Library/Caches to the list of shared paths? Or f.e. disable this feature by default for a transitional period of time with the warning emerging when someone tries to bind mount directories that will not be available when this feature becomes enabled?

[stephen-turner]

The reason we disabled it in the first place was because of scary security warnings as reported in #4975. We are trying to work round that, but as they are OS level messages we don't have much control over them. There are also performance concerns if Docker can sync its own working directories because of the risk of infinite loops, but those are more under our control.

We realise the current situation is not satisfactory for people who do want to mount subdirectories of ~/Library, even if it has fixed #4975 for the much larger number of people who blindly mount the whole of ~. So today we are looking at (1) can we exclude an even smaller set of directories (but I bet there's someone somewhere who wants to mount ~/Library/Reminders for some reason); (2) can we only mount them if they are explicitly read from within the container (and can we do that in a performant way?); (3) can we exclude them by default but allow the user to turn them on?

[dlgoodchild]

@dlgoodchild please correct me if I am wrong but despite being experimental "Use gRPC FUSE" feature is enabled by default for Docker for Mac:

You're absolutely right, and that is probably the root of the problem. Who in their right mind would make an experimental feature the default? It's crazy that it just slips in there and breaks everything. Maybe it should have been a prompt post-install like "We recommend this feature, but it's experimental ....blah blah".

It's pretty much the same as forcing everyone onto a beta version. Pretty uncool.

[dee-kryvenko]

@dlgoodchild I do not think I was overreacting. First of all as already being mentioned - labeling this feature experimental means nothing to me as experimental features enabled by default does not sound so experimental to me.

But what's even more important and what's not being mentioned so far - we all know how it goes with all the experimental features. They eventually becomes a new norm and the old norm being slowly pushed out and retired. Even if they might not have the intention to retire the old mechanism right now, but as you mentioned yourself - it always comes to prioritizing limited resources so I do not think the old mechanism will be getting too much attention going forward and will naturally be retired in a some time.

And the fun part - I did tried to disable gRPC FUSE on 3.0.1 and I faced another issue - files updated from the host was not updating in the container. Had to downgrade since I don't have time to investigate this issue and file a new ticket right now, it might very well be that I just faced another known issue like moby/moby#15793.

Whether there are technical reason behind not being able to use ~/Library with gRPC FUSE or not seems irrelevant to me. Docker can't not be having an ability to mount ~/Library, it would just make docker completely unusable. This should be either fixed or gRPC FUSE is just not the right solution here. Not to mention that I do have gRPC FUSE enabled in 2.5.0.1 and it works just fine - this evidence doesn't support the claim that there is any technical limitations.

And finally, adding weight and trying to indicate exactly how much there is an evident demand for this out there and how extreme the priority is - is exactly what I am trying to do here. It might be a little too late to be doing that when this not-so-experimental feature becomes not-at-all-experimental.

@stephen-turner, quite honestly I am not even sure why #4975 was even considered. This is clearly an OS issue. You mount stuff - OS asking permissions. It's just the way it is. Maybe it's just me but I would expect Docker users to know exactly why such a warnings pops up and not having any problems with them. It's not like Docker users are sales managers and bank clerks, or are they? We all here getting the basic concepts of OS and FS and security permissions...

Anyway, whatever you decide to do with it - just remember you can't be making assumptions like "no, the users certainly would never need to mount this particular directory". They most certainly will. Current behavior is not just "not satisfactory" - it's making Docker completely useless for a subset of DevOps community that packs their toolchain as docker images - and how big exactly that subset is I don't know but I want to believe it's an ultimate majority at the very least.

[stephen-turner]

It is absolutely our intention to replace osxfs with gRPC Fuse, and we have stated so many times. gRPC Fuse is now the default because we consider it superior overall, although they each have issues that the other doesn't have, which is why we have retained the toggle for the moment. To be honest it probably shouldn't be in the experimental panel any more now that it's graduated to be the default, but we don't have a better place for it and so kept it there.

The symptom with #4975 is that someone bind mounts the whole of $HOME — and not even typing it explicitly, they are using a web framework that does that internally — maybe they don't even understand Docker very well but just know the runes to make their project compile — and then several hours later, when they're not even using Docker, a warning pops up saying that Docker wants to read their Reminders. This is scary, and they don't understand it. So it absolutely needs to be fixed.

Anyway, I think I've been clear that we're fixing this as fast as we can. We just need to find the right solution that doesn't make things worse for the much larger web developer audience. Our current plan is what I listed as (3) above: ~/Library (or a subdirectory) would be excluded by default if you only share the home directory, but you can explicitly include it to be shared.

[dee-kryvenko]

To be honest I probably do not understand the proposed solution, because if I do - it sound to me like Docker is going to have in that case an include list, from which there's going to be exclude list, which in turn will have another include sub-list. That doesn't feel right for a mature product... not very intuitive. Most likely error prone and will cause you troubles in the future.

Why wouldn't docker just pro-actively explain it's behavior to the user? Docker already has a warning with explanation why the user gonna need to enter credentials in the next screen. Docker can just expand on this existing pattern and proactively detect that some of the user actions may trigger these OS warnings and then pop up a warning with explanation that that could happen and the reasons why. This would sound like the least intrusive approach.

[CourteousCoder]

Updating to 3.0.1 didn't fix this for me.

[norrisjeremy]

I too have to downgrade from 3.0.1 to 2.5.0.1 in order to have functioning bind mounts again.

[vincentmrg]

If I understand correctly we have the choice between the version which jokes on the volume mounts and the version with the hyperkit which eats all the cpu.

[stannnous]

Confirming this is still broken in 3.0.1. The disable "Use gRPC Fuse for file sharing" workaround continues to work though.

[n3dj0]

Confirming this is still broken in 3.0.1. The disable "Use gRPC Fuse for file sharing" workaround continues to work though.

+1, it is still broken and a factory reset did also NOT help.

[stephen-turner]

We don't need any more confirmations if it's ~/Library that is blocked. (If anything else is still blocked, then please do let us know, but we don't think that's the case).

Our plan has changed since my last message. We're now planning to do what @dee-kryvenko suggested at #5115 (comment) (thank you, Dee), and issue a warning if someone tries to share the whole of their home directory. I expect it will annoy quite a lot of people as this is a common idiom, especially if they're not sharing it directly but using a framework that shares it, but on reflection it seems like user education probably is the way forward.

[norrisjeremy]

We don't need any more confirmations if it's ~/Library that is blocked. (If anything else is still blocked, then please do let us know, but we don't think that's the case).

Our plan has changed since my last message. We're now planning to do what @dee-kryvenko suggested at #5115 (comment) (thank you, Dee), and issue a warning if someone tries to share the whole of their home directory. I expect it will annoy quite a lot of people as this is a common idiom, especially if they're not sharing it directly but using a framework that shares it, but on reflection it seems like user education probably is the way forward.

The issue I am having is that I am no longer able to reliably use qemu-nbd + kpartx to mount disk image files inside a container that sit in bind volumes & this occurs regardless of the gRPC Fuse setting in 3.x.

I frequently end up with invocations of qemu-nbd to attach disk image files from within a bind volume that hang forever, or if the qemu-nbd invocation randomly succeeds, an invocation of kpartx to find the partitions in the disk image fail with read error, sector 0 + llseek error.

This does not occur with 2.5.0.1 (both with gRPC Fuse enabled or disabled).

[stephen-turner]

OK, this should be completely fixed in 3.0.2. You can now share ~/Library again.

[souravchandra]

Docker desktop is stuck during start after I upgrade to 3.0.2

Mac version

[dee-kryvenko]

@stephen-turner seems 3.0.2 is working for me again. Didn't see any warnings though so far.

[docker-robott]

Closed issues are locked after 30 days of inactivity.
This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle locked