Developing with and hosting upon OpenShift

Developing with and hosting upon OpenShift

Istio includes a configuration profile to function using OpenShift.

Installing the open source Istio control plane upon OpenShift clusters

Istio provides instructions for using OpenShift.

Istio includes an OpenShift profile. It is not sufficient to merely istioctl install --set profile=openshift. There are steps to take before and after installing that are shown in the document referenced above.

The only difference in the 1.8.0 OpenShift profile is the Istio CNI configuration.

(To understand the differences between profiles I use sdiff <(istioctl profile dump default) <(istioctl profile dump openshift).)

Installing the Istio Control Plane on OpenShift in an external configuration with non-OpenShift data plane clusters.

Istio can be configured to run the control plane in a different cluster than the dataplane. In Istio 1.8.x, the configuration of CNI is normally the same between the control and data planes.

To set up the OpenShift control plane cluster for Istio, follow the Istio OpenShift instructions, except for the istioctl install --set profile=openshift step, on the control plane cluster.

When installing Istio on the control plane using external-istiod.yaml, but with the CNI settings desired for OpenShift, create CNI configuration IstioOperator override YAML. At the step to install the external control plane, supply the override YAML in addition to external-istiod.yaml, like this:

cat > external-openshift-iop.yaml <<EOF
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  components:
    cni:
      enabled: true
      namespace: kube-system
  values:
    cni:
      cniBinDir: /var/lib/cni/bin
      cniConfDir: /etc/cni/multus/net.d
      chained: false
      cniConfFileName: "istio-cni.conf"
      excludeNamespaces:
       - istio-system
       - kube-system
      logLevel: info
EOF
istioctl install -f external-openshift-iop.yaml -f external-istiod.yaml --context="${CTX_EXTERNAL_CLUSTER}"

IBM Cloud instructions to retrieve EXTERNAL_ISTIOD_ADDR and SSL_SECRET_NAME

To use external control planes, you need EXTERNAL_ISTIOD_ADDR and SSL_SECRET_NAME. To get these on IBM Cloud:

export INGRESS_IP=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
echo INGRESS_IP is $INGRESS_IP
export EXTERNAL_ISTIOD_ADDR=$(ibmcloud cs nlb-dns ls --cluster ${EXTERNAL_CLUSTER_NAME} --output json | jq --raw-output ".[]  | select(.nlbIPArray | index(\"$INGRESS_IP\")) | .nlbHost")
echo EXTERNAL_ISTIOD_ADDR is $EXTERNAL_ISTIOD_ADDR

If EXTERNAL_ISTIOD_ADDR has no value, you need to add an NLB for the ingress gateway:

ibmcloud ks nlb-dns create classic --cluster ${EXTERNAL_CLUSTER_NAME} --ip ${INGRESS_IP} --secret-namespace istio-system

# If there is >1 secret we can't scrape the secret name from `kubectl get secrets` because there is no owner pointing back to the nlb-dns.
# kubectl -n istio-system get secrets --field-selector type=kubernetes.io/tls -o json | jq --raw-output '.items[0].metadata.name'
export SSL_SECRET_NAME=$(echo ${EXTERNAL_ISTIOD_ADDR} | sed 's/\([^\.]*\).*/\1/')
echo SSL_SECRET_NAME is $SSL_SECRET_NAME

Troubleshooting Istio on OpenShift

As usual with OpenShift, you'll need oc login --token=$TOKEN --server=$SERVER for kubectl and istioctl access.

When Istio is installed in an external control plane configuration it will not use the istio-system namespace. Add --context="${CTX_EXTERNAL_CLUSTER}" -n external-istiod parameters to the kubectl troubleshooting commands to look for problems.

If you forget the additional OpenShift steps, the Istio control plane pods are not created. kubectl -n istio-system get rs shows no instances created. kubectl -n istio-system describe rs shows "securityContext.securityContext.runAsUser: Invalid value: 1337: must be in the ranges ...". You may also see errors such as ResourceSet descriptions showing "Warning FailedCreate 2s (x14 over 43s) replicaset-controller Error creating: Internal error occurred: failed calling webhook "sidecar-injector.istio.io": Post https://istiod.istio-system.svc:443/inject?timeout=30s: service "istiod" not found"

Use oc get scc and oc describe scc anyuid to inspect configuration.

OSX / MacOS

Fix "“oc” cannot be opened because the developer cannot be verified" errors using System Preferences, Security, General. Look for "'oc' was blocked ..." and click "Allow Anyway". Then run it again, and click "Open".