Splunk Operator: No way to start searchheads independently from CM

[yaroslav-nakonechnikov]

Please select the type of request

Bug

Tell us more

Describe the request
There is a need to start cluster of searcheads wihtout pointing to any cluster manager and indexers.
and as it was suggested in #1227 , i'm creating that issue.

Expected behavior
Removing cluster manager ref will be ignored and searchhead cluster will be started without any issues.

Splunk setup on K8S

• aws eks

Reproduction/Testing steps

• just remove cluster manager ref from config and it will fail

[logsecvuln]

could someone please take a look into this @vivekr-splunk @jryb @kumarajeet

[vivekr-splunk]

@yaroslav-nakonechnikov we will look at this issue and get back to you

[akondur]

Hey @yaroslav-nakonechnikov , can you try this with the new Splunk operator version 2.5.1?

[yaroslav-nakonechnikov]

@akondur so, am i understand correct, that to start searchhead independently, it should be possible just remove clusterManagerRef?

or something else is needed?

[akondur]

Hey @yaroslav-nakonechnikov , I was able to successfully deploy a SHC without a clusterManagerRef:

SHC Spec without clusterManagerRef

apiVersion: enterprise.splunk.com/v4
kind: SearchHeadCluster
metadata:
  name: example
  namespace: splunk-operator
  finalizers:
  - enterprise.splunk.com/delete-pvc

SHC deployed and running:

bash % kubectl get shc
NAME      PHASE   DEPLOYER   DESIRED   READY   AGE
example   Ready   Ready      3         3       5m11s
bash % kubectl get pods
NAME                                                 READY   STATUS    RESTARTS   AGE
splunk-example-deployer-0                            1/1     Running   0          5m21s
splunk-example-search-head-0                         1/1     Running   0          5m21s
splunk-example-search-head-1                         1/1     Running   0          5m20s
splunk-example-search-head-2                         1/1     Running   0          5m20s
splunk-example-standalone-0                          1/1     Running   0          48m
splunk-operator-controller-manager-56b66f6f4-gtcns   2/2     Running   0          54m

To understand the issue better, can you list the exact steps you used to hit the issue? Do you first connect it to an indexer cluster via clusterManagerRef and then remove it? If yes, do you also delete the indexer cluster while doing so? Please let us know so we can triage this issue better.

[akondur]

Hey @yaroslav-nakonechnikov , can you give us some more details here so we can make progress on this issue?

[yaroslav-nakonechnikov]

@akondur i can't do check without solving #1293
as 2.5.x is causing issues, which we can't debug.