How to only create volumes on some nodes?

[R0flcopt3r]

I want to only store data on the nodes that have fast and plenty storage. I can achieve this by disabling scheduling on the nodes I don't want storage on. But I can't seem to figure out how to do this programmatically. I'm using the helm chart from rancher apps, longhorn version 1.5.3.

I'm also wondering what the difference between the longhorn manager and driver is? In the helm chart i can specify nodeSelector for both. A node with only driver can't seem to access longhorn volumes, but a node with only manager can, regardless of which node the volume exist on. However any node with the longhorn manager is schedulable.

If i don't have longhorn manager running on a node with a pod that tries to access i get a 404 error.

[derekbit]

I want to only store data on the nodes that have fast and plenty storage. I can achieve this by disabling scheduling on the nodes I don't want storage on. But I can't seem to figure out how to do this programmatically. I'm using the helm chart from rancher apps, longhorn version 1.5.3.

You can leverage the python client and examples.

[R0flcopt3r]

But there is no support for this via a configmap or helm? Is marking nodes as unschedulable even the "correct" solution? To me this feels like some really basic functionality that should be officially supported some way.

[derekbit]

Sorry, I'm confused by your original question do this programmatically and configmap in the reply...

If you want to disable scheduling on the nodes, there are two ways

• Update the nodes programmatically by the python client as I mentioned above)
• Edit node.longhorn.io resources

As for configmap method, Longhorn doesn't support it.

[R0flcopt3r]

Programmatically was a poor choice of word. What I meant was "without clicking through the GUI".

When we add a new node to the cluster, we need to be able to control whether it's schedulable or not, or more precise if it holds data or not. Our nodes are designed to be short lived, so this needs to be stable and automatic. The best way would be if it was a label on the node, since that's very easy for us to do.

If we have to edit the node.longhorn.io resources we'd need some workload checking for new nodes, changes in longhorn etc, and then react to them. This sounds very prone to race issues.

[c3y1huang]

Have you checked out Tip: Set Longhorn To Only Use Storage On A Specific Set Of Nodes?

[R0flcopt3r]

Thanks! This is exactly it! Now i feel real dumb, and i can't understand how I missed this one...