Access Denied trying to mount newly created SMB share

[mglazenborg]

What happened:
Access is denied trying to mount newly created SMB share. Occurs on both Linux & Windows nodes.

Windows Pods don't start and give the following error:

Error: failed to generate container "cf9cba469bc2af5d586392d9baf62d59ef4d3f99f2711eccc896588f163ccf73" spec: failed to generate spec: failed to stat "c:\\var\\lib\\kubelet\\pods\\c9e673b8-df2b-464c-80fb-2465dfdd5994\\volumes\\kubernetes.io~csi\\<PV-NAME>\\mount": CreateFile c:\var\lib\kubelet\pods\c9e673b8-df2b-464c-80fb-2465dfdd5994\volumes\kubernetes.io~csi\<PV-NAME>\mount: Access is denied.

Linux Pods do start but return the following error when trying to view the folder:

ls: reading directory '/smbshare/': Permission denied

This only occurs when existing SMB shares are mounted under the same user that is being used for the new share.

What you expected to happen:
The file share should mount with the correct permissions.

How to reproduce it:
Mount an existing file share with an account. Create a new file share and assign the same account rights to access it.
Lastly mount the newly created file share and the error should occur.

Anything else we need to know?:
Removing all the file shares and then recreating them does fix this issue, but is not a solution as removing them all each time a new one is needed is not an option.

Environment:

• CSI Driver version: 1.14
• Kubernetes version (use kubectl version): 1.28.5
• OS (e.g. from /etc/os-release): Ubuntu 22.04.3 LTS & Windows Server 2019
• Kernel (e.g. uname -a): 5.15.0-1054-azure
• Install tools: Helm

[andyzhangx]

does mounting file share manually on the linux or windows node work?

[mglazenborg]

Unfortunately I don't have direct access to the nodes, but using a debug container I am able to mount the share manually.

[andyzhangx]

try using kubectl-enter to ssh to the node, and then run mount command:

sudo wget
https://raw.githubusercontent.com/andyzhangx/demo/master/dev/kubectl-enter
sudo chmod a+x ./kubectl-enter
./kubectl-enter <node-name>

[mglazenborg]

Looking at that script, the debug container I use does the exact same and as said before that does work. However, mounting it using the driver in a PV/PVC doesn't work.