CreateComputeSystem fails with "Do not attach the filter to the volume at this time." when a WinFsp volume is used as the host path for a mount #498
Comments
|
Oh and if it helps, here's all the info from The NTPTFS mount is definitely that last entry there, with a status of |
|
Interestingly enough, using a UNC path kind of works more in the sense that the container can be created, but accessing the mounted path fails with Access Denied: apiVersion: v1
kind: Pod
metadata:
name: uefs-test-attach
namespace: default
spec:
nodeSelector:
kubernetes.io/hostname: computer-name
containers:
- name: uefs-test-attach
image: mcr.microsoft.com/powershell:lts-7.2-nanoserver-ltsc2022
stdin: true
tty: true
resources: {}
volumeMounts:
- name: pkg-vol
mountPath: "C:\\DoesNotMatter"
terminationGracePeriodSeconds: 0
volumes:
- name: pkg-vol
hostPath:
path: "\\\\foo\\bar"UNC paths are also less than ideal due to the requirement that they be associated with a drive letter (and we might want more than 24-ish mounts if there's a non-trivial number of containers running on the host). |
|
Thinking about this more, I think a viable solution might look something like this:
While this wouldn't directly fix the issue of not being able to mount at container creation time, it would provide a viable pathway to using WinFsp inside containers where the filesystem needs to be served from a process on the host (without incurring the performance penalty of sending all file I/O operations over loopback or a named pipe). |
|
Oh hmm, perhaps there's a better way of getting the behaviour closer to what is expected of the volume mounting API:
|
|
Apologies, but I have not had time to look into this as I am very busy with my new hypervisor project VirtualMetal. I am now coming close to an initial release on that other project, and I will be able to catch up here soon. |
|
@billziss-gh Just wanted to check in and see how far off your VirtualMetal work is to being finished so this can be looked at, since that will guide me on whether or not it's worth attempting to implement a patch for this myself. |
|
@hach-que I am not sure. The VirtualMetal hypervisor is now able to boot and debug minimal Linux+Busybox, which is probably good enough for an initial release. However this is just a small part of the capabilities I want it to have so I keep hacking at it. I have not even had the time to understand the problem you are having. Have you found a sensible patch or work around? |
|
I haven't even started looking at what a patch would look like other than the general design I came up with in #498 (comment). I'd have to learn how all the WinFsp kernel bits fit together and get a debugging VM set up. I figured if this was something you were going to get a chance to look at within a few weeks you'd probably be able to solve it much much faster than I could learn the kernel code and write a patch, but if the ETA is still unknown it's probably worth it for me to start figure out what a patch would look like. |
Bug Report
I'm not actually sure if this is actionable on the WinFsp side of things, but since bugs in Windows Containers probably don't have a quick turnaround, I'm hoping there's something that can be done in WinFsp to make this work.
Basically, when you use WinFsp and make a normal mount on the host (just with ntptfs or w/e) with something like this:
And then you try to use
C:\SomeTargetPathOnTheHostas the host path for a mount in a container,CreateComputeSystemthen fails with "Do not attach the filter to the volume at this time.". I'm pretty sure this is because HCS internally is trying to attach a minifilter or something to the volume, but because it's not a normal block device backed by a VHD/WinSpd/disk controller, it doesn't actually work.(For context, using a non mount manager mount also fails, but because containerd fails to resolve the junction to a volume name)
How to Reproduce
The steps to reproduce this are unfortunately not trivial, only because you need a way of creating Windows Containers.
First up, get
ntptfsrunning with a simple mapping as shown above.If you're running Kubernetes on the Windows node, you could create a host mapping like so:
An example of the spec sent to CreateComputeSystem is shown in the issue I filed in the Windows Containers repo, in case you want to create the container manually: microsoft/Windows-Containers#335
Behaviors
The container should be created successfully, and ideally HCS's CreateComputeSystem wouldn't try to apply a filter on top of a WinFsp volume. I'm hoping there's some kernel magic WinFsp can do to recognise the attempt at attaching a filter onto a volume it controls, and either pretend like the attachment was successful, or use the attempted attachment to make the volume visible to silos on the system, just so the container can be created.
Environment
The text was updated successfully, but these errors were encountered: