driver "windowsfilter" failed to remove root filesystem: failed to detach VHD Error while Creating Business Central Docker Image for Indian Version

[Sandygitl]

Hi All,

I am getting below error while installing Business Central Indian Version Image. Tried this with different laptops but getting same error

PS C:\windows\system32> install-module navcontainerhelper -force 
$artifactUrl = Get-BCArtifactUrl -storageAccount bcpublicpreview -country in -select Latest -type Sandbox
New-BcContainer `
-accept_eula `
-containerName BC-IN-PP `
-artifactUrl $artifactUrl `
-auth NavUserPassword `
-updateHosts `
-assignPremiumPlan `
-includeAL
NavContainerHelper is version 0.7.0.26
NavContainerHelper is running as administrator
Host is Microsoft Windows 10 Enterprise - 1909
Docker Client Version is 19.03.8
Docker Server Version is 19.03.8
Removing container BC-IN-PP
Removing BC-IN-PP from host hosts file
Removing C:\ProgramData\NavContainerHelper\Extensions\BC-IN-PP
Fetching all docker images
Using image mcr.microsoft.com/dynamicsnav:10.0.18363.1082-generic
Downloading application artifact /sandbox/17.0.16993.0/in
Downloading C:\Users\sandeep.mishra07\AppData\Local\Temp\5325f13d-b684-4805-ba6b-38e5f9427e33.zip
Unpacking application artifact to tmp folder using Expand-Archive
Downloading platform artifact /sandbox/17.0.16993.0/platform
Downloading C:\Users\sandeep.mishra07\AppData\Local\Temp\50982391-44b6-467b-af19-164cdd4dc2cd.zip
Unpacking platform artifact to tmp folder using Expand-Archive
Downloading Prerequisite Components
Downloading c:\bcartifacts.cache\sandbox\17.0.16993.0\platform\Prerequisite Components\Open XML SDK 2.5 for Microsoft Office\OpenXMLSDKv25.msi
Downloading c:\bcartifacts.cache\sandbox\17.0.16993.0\platform\Prerequisite Components\IIS URL Rewrite Module\rewrite_2.0_rtw_x64.msi
Downloading c:\bcartifacts.cache\sandbox\17.0.16993.0\platform\Prerequisite Components\DotNetCore\DotNetCore.1.0.4_1.1.1-WindowsHosting.exe
Creating Container BC-IN-PP
Version: 17.0.16993.0-IN
Style: sandbox
Platform: 17.0.16974.0
Generic Tag: 0.1.0.21
Container OS Version: 10.0.18363.1082 (1909)
Host OS Version: 10.0.18363.1082 (1909)
Using locale en-IN
Using process isolation
Disabling the standard eventlog dump to container log every 2 seconds (use -dumpEventLog to enable)
Files in C:\ProgramData\NavContainerHelper\Extensions\BC-IN-PP\my:
- AdditionalOutput.ps1
- MainLoop.ps1
- SetupNavUsers.ps1
- SetupVariables.ps1
- updatehosts.ps1
Creating container BC-IN-PP from image mcr.microsoft.com/dynamicsnav:10.0.18363.1082-generic
7b6b84347202b4a92096a0a551ecbedc73ed1fa5a51ccad560a0b9b8c7076780
Docker : Error response from daemon: container 7b6b84347202b4a92096a0a551ecbedc73ed1fa5a51ccad560a0b9b8c7076780: driver "windowsfilter" failed to remove root filesystem: failed to 
detach VHD: The device is not ready.: rename C:\ProgramData\Docker\windowsfilter\7b6b84347202b4a92096a0a551ecbedc73ed1fa5a51ccad560a0b9b8c7076780 
C:\ProgramData\Docker\windowsfilter\7b6b84347202b4a92096a0a551ecbedc73ed1fa5a51ccad560a0b9b8c7076780-removing: Access is denied.
At C:\Program Files\WindowsPowerShell\Modules\navcontainerhelper\0.7.0.26\HelperFunctions.ps1:124 char:17
+                 Docker rm $out -f
+                 ~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (Error response ...cess is denied.:String) [], RemoteException
    + FullyQualifiedErrorId : 

> `NativeCommandError```

[freddydk]

This error is very machine specific - has nothing to do with the Indian image.
You probably also cannot run a standard Windows Server Core image nor any other Business Central images right?
Have you ever been able to run any container on that machine?
Very surprised if you got the same error on multiple machines - then they all must have some software installed, which causes issues.
These docker errors are very hard for me to help troubleshoot as they really has nothing to do with Business Central or ContainerHelper.

[Sandygitl]

This error is very machine specific - has nothing to do with the Indian image.
You probably also cannot run a standard Windows Server Core image nor any other Business Central images right?
Have you ever been able to run any container on that machine?
Very surprised if you got the same error on multiple machines - then they all must have some software installed, which causes issues.
These docker errors are very hard for me to help troubleshoot as they really has nothing to do with Business Central or ContainerHelper.

Hi Freddyk,

Thanks for the update.

I am able to run other Business Central Image. Already running container with using below image :
mcr.microsoft.com/businesscentral/sandbox:16.2.13509.13619-us.

[Sandygitl]

This error is very machine specific - has nothing to do with the Indian image.
You probably also cannot run a standard Windows Server Core image nor any other Business Central images right?
Have you ever been able to run any container on that machine?
Very surprised if you got the same error on multiple machines - then they all must have some software installed, which causes issues.
These docker errors are very hard for me to help troubleshoot as they really has nothing to do with Business Central or ContainerHelper.

Hi Freddyk,

Thanks for the update.

I am able to run other Business Central Image. Already running container with using below image :
mcr.microsoft.com/businesscentral/sandbox:16.2.13509.13619-us.

Getting error with this particular Image only.

[freddydk]

There is a huge difference in running an image (mcr.microsoft.com/businesscentral/sandbox:16.2.13509.13619-us) and running with artifacts (like what you are doing now).
How much memory do you have in the machine?

[Sandygitl]

There is a huge difference in running an image (mcr.microsoft.com/businesscentral/sandbox:16.2.13509.13619-us) and running with artifacts (like what you are doing now).
How much memory do you have in the machine?

16 GM RAM is there in the machine.
Also, Is there any image for IN localization instead of artifact.
Thanks!

[Sandygitl]

There is a huge difference in running an image (mcr.microsoft.com/businesscentral/sandbox:16.2.13509.13619-us) and running with artifacts (like what you are doing now).
How much memory do you have in the machine?

16 GM RAM is there in the machine.
Also, Is there any image for IN localization instead of artifact.
Thanks!

I am not able to find any image other than this particular artifact.

[freddydk]

Images are history - artifacts is the future
If the above is the full output, the error comes before any initialization of the container is done and is likely because your machine might be low on resources.
Just to be sure - I have just run the above artifacts - they work fine.
With docker errors like this I typically ask people to update to latest versions and reset to factory defaults (which will remove existing images/containers).

[siegeon]

FWIW - I constantly run into the detach issue, may like 1 in 10. For us, it does not matter what the image is at all.

Docker : Error response from daemon: container *: driver "windowsfilter" failed to remove root filesystem: failed to
detach VHD: The device is not ready.: rename C:\ProgramData\Docker\windowsfilter*
C:\ProgramData\Docker\windowsfilter*-removing: Access is denied.

[lippertmarkus]

We found that it's related to Defender that sometimes is blocking either deletion or container creation (noticeable when the container status remains in "created" or "removing").
Adding the Docker directories as exclusion didn't help, but the following sledhammer approach works for us:

Set-MpPreference -DisableIntrusionPreventionSystem $true -DisableIOAVProtection $true -DisableRealtimeMonitoring $true -DisableScriptScanning $true -EnableControlledFolderAccess Disabled -EnableNetworkProtection AuditMode -Force -MAPSReporting Disabled -SubmitSamplesConsent NeverSend

Needs a restart aftewards.

[marknitek]

Well this renders the Defender useless i would say 😂? Not very corporate friendly...

Unfortunately i (and most of my colleagues) have the same issue from time to time (not always, so it's hard to troubleshoot).
This issue remains after the upgrade to windows 11 (which i mainly did to resolve all docker related issues 🙄)

And Hyper-V is not an option... It just consumes to many resources, so this issue should be resolved somehow...

@freddydk do you have any more insights to this? I'am a bit surprised not more people are reporting this, because as said, in my company multiple people are suffering from it (since 1-2 weeks)

[ted-duncan]

Also seeing this error on Windows Server 2022 Datacenter, Docker details are as follows:
Client: Mirantis Container Runtime
Version: 20.10.9
API version: 1.41
Go version: go1.16.12m2
Git commit: 591094d
Built: 12/21/2021 21:34:30
OS/Arch: windows/amd64
Context: default
Experimental: true

Server: Mirantis Container Runtime
Engine:
Version: 20.10.9
API version: 1.41 (minimum version 1.24)
Go version: go1.16.12m2
Git commit: 9b96ce992b
Built: 12/21/2021 21:33:06
OS/Arch: windows/amd64
Experimental: false

[marknitek]

Can confim that disabling Realtime Protection solves the issues (multiple clients tested). But as said, its not a real solution...

[olljanat]

We found that it's related to Defender that sometimes is blocking either deletion or container creation (noticeable when the container status remains in "created" or "removing"). Adding the Docker directories as exclusion didn't help

Btw. Did you tried to exclude dockerd.exe process instead of (or together with) directories? That should do most of the trick and let docker do the clean up.

[freddydk]

FYI - I found that on my machine, HyperV isolation is much faster than process isolation - probably because of defender or like.