Problem with Telephony Network Service v0.5.0 deployment on K3S

[KaioKei]

Telephony Network Service deployment on K3S with v0.5.0 is broken

Overview

Hello,

I am working on a usecase to deploy multi-clustered applications with TOSCA on K3S, so I am trying to reproduce the Telephony Network Service example, but I did not managed to succeed yet.

Issue

During the deployment, the service is blocked in 'Instantiating' state :

$ turandot service deploy telephony-network-service --template=telephony-network-service --kubeconfig ~/.kube/tns-central --namespace turandot --input namespace=turandot
$ turandot service list --kubeconfig ~/.kube/tns-central --namespace turandot

Name                      State         Mode Inputs Outputs
═════════════════════════╤═════════════╤════╤══════╤═══════
telephony-network-service│Instantiating│    │      │       
═════════════════════════╧═════════════╧════╧══════╧═══════

Its description brings no more information :

turandot service describe telephony-network-service --kubeconfig ~/.kube/tns-central --namespace turandot
Name: telephony-network-service
ServiceTemplate:
  Indirect:
      Registry: default
          Name: turandot/service-templates-telephony-network-service:latest
          InstantiationState: Instantiating
          CloutPath: /cache/clout/turandot-telephony-network-service-cd586faf-30df-4795-b75e-cf3e10b080d5.yaml
          CloutHash: 7GwmvvHG7nT9Q+5/L9T3mg==
          Mode:

The kubernetes events are a little more explicit :

$ kubectl get events -n turandot

3m22s       Warning   InstantiationError   service/telephony-network-service          Service instantiation error: could not parse policies:...
3m18s       Warning   InstantiationError   service/telephony-network-service          Service instantiation error: could not parse policies:...
3m17s       Normal    Compiled             service/telephony-network-service          Service template compiled successfully

More precisely :

$ kubectl -n turandot describe service.turandot.puccini.cloud/telephony-network-service
  Warning  InstantiationError  9m17s (x7 over 9m30s)  Turandot  Service instantiation error: could not parse policies:
central-pbx:             
  - properties:             
      instantiable: true 
      profile:           
        name: large         
      sites:            
        - central           
      substitutable: true
      substitutionInputs:
        namespace: workspace
      virtualizable: true   
    type: provisioning   
data-plane:
  - properties:
      instantiable: true
      profile:              
        name: simple     
      sites:             
        - edge              
      substitutable: true
      substitutionInputs:
        ip-prefix: 192.168.3                                                                                                          
        name: data-plane
        namespace: workspace
      virtualizable: true
    type: provisioning   
  - properties:             
      instantiable: true
      profile:              
        name: simple     
      sites:             
        - central           
      substitutable: true
      substitutionInputs:
        ip-prefix: 192.168.2
        name: data-plane
        namespace: workspace
      virtualizable: true
    type: provisioning   
edge-pbx:                
  - properties:             
      instantiable: true 
      profile:
        name: small
      sites:            
        - edge              
      substitutable: true
      substitutionInputs:
        namespace: workspace
      virtualizable: true
    type: provisioning
  Warning  InstantiationError  9m13s (x5 over 9m30s)  Turandot  Service instantiation error: could not parse policies:                
central-pbx:
  - properties:
      instantiable: true 
      profile:           
        name: large         
      sites:            
        - central           
      substitutable: true
      substitutionInputs:
        namespace: workspace
      virtualizable: true
    type: provisioning  
data-plane:   
  - properties:    
      instantiable: true
      profile:
        name: simple     
      sites:             
        - central           
      substitutable: true
      substitutionInputs:
        ip-prefix: 192.168.2
        name: data-plane
        namespace: workspace
      virtualizable: true
    type: provisioning
  - properties:
      instantiable: true
      profile:
        name: simple
      sites:
        - edge
      substitutable: true
      substitutionInputs:
        ip-prefix: 192.168.3
        name: data-plane
        namespace: workspace
      virtualizable: true
    type: provisioning
edge-pbx:
  - properties:
      instantiable: true
      profile:
        name: small
      sites:
        - edge
      substitutable: true
      substitutionInputs:
        namespace: workspace
      virtualizable: true
    type: provisioning
  Normal  Compiled  4m30s (x245 over 9m30s)  Turandot  Service template compiled successfully

I do not understand what is happening here. One of the logs says Turandot compiled the TOSCA template successfully, with parsing errors, but the service cannot be deployed. How can I get more debug information ?

Walkthrough

I installed two K3S clusters (one-node installation) on two different hosts :

• host tns-central :
  • k3s site tns-central
• host tns-edge
  • k3s site tns-edge

I am using 2 kubeconfig files : one for the cluster tns-central and one for the cluster tns-edge, due to the K3S installation.

I installed Multus on both of the cluster. I used the 'thick' daemonset manifest and I modified it to configure the proper path to the cni configurations for K3S:

volumes:
  - name: cni
    hostPath:
      path: /var/lib/rancher/k3s/agent/etc/cni/net.d
  - name: cnibin
    hostPath:
      path: /var/lib/rancher/k3s/data/current/bin

and args :

containers:
  - name: kube-multus
    image: ghcr.io/k8snetworkplumbingwg/multus-cni:thick
    command: ["/usr/src/multus-cni/bin/multus-daemon"]
    args:
      - "-cni-version=0.3.1"
      - "-cni-config-dir=/host/etc/cni/net.d"
      - "-multus-autoconfig-dir=/host/etc/cni/net.d"
      - "-multus-log-to-stderr=true"
      - "-multus-log-level=verbose"
      - "--multus-kubeconfig-file-host=/var/lib/rancher/k3s/agent/etc/cni/net.d/multus.d/multus.kubeconfig"

I installed Kubevirt on tns-central.

A Reposure registry is mandatory for turandot services and templates in K3S even though it appears to be missing in Turandot v0.5.0 documentation (it concerns other distribution like Kind). Can you confirm ?
If I install the turandot operator without reposure I get an error saying :

$ turandot template list --kubeconfig $HOME/.kube/tns-central -n turandot
registries.reposure.puccini.cloud "default" not found

So I installed Reposure using the reposure client v0.1.4 in tns-central in the namespace turandot :

reposure operator install --kubeconfig $HOME/.kube/tns-central -n turandot
# registry installation with simple provider
kubectl apply --filename=https://github.com/jetstack/cert-manager/releases/download/v1.3.1/cert-manager.yaml
reposure simple install --kubeconfig $HOME/.kube/tns-central --authentication -n turandot
reposure registry create default --provider=simple --kubeconfig $HOME/.kube/tns-central -n turandot

Then I installed the Turandot operator using the cli v0.5.0 :

turandot operator install --kubeconfig $HOME/.kube/tns-central -n turandot --site=tns-central

I delegated tns-edge to tns-central.

turandot delegate set tns-edge --kubeconfig $HOME/.kube/tns-central --delegate-kubeconfig=$HOME/.kube/tns-edge --namespace turandot

I downloaded the Turandot repository and built the container images as mentioned in this example instructions :

git clone https://github.com/tliron/turandot && cd turandot/examples/telephony-network-service
./scripts/build-asterisk-cnf-container-image
sudo ./scripts/build-asterisk-vnf-container-image

I registered the templates using the proper kubeconfig and namespace :

turandot template register telephony-network-service --file=$HOME/turandot/dist/telephony-network-service.csar -n turandot --kubeconfig ~/.kube/tns-central
turandot template register simple-data-plane --file=$HOME/turandot/dist/simple-data-plane.csar -n turandot --kubeconfig ~/.kube/tns-central
turandot template register asterisk-cnf --file=$HOME/turandot/dist/asterisk-cnf.csar -n turandot --kubeconfig ~/.kube/tns-central
turandot template register asterisk-vnf --file=$HOME/turandot/dist/asterisk-vnf.csar -n turandot --kubeconfig ~/.kube/tns-central

Finally, I deployed the service :

turandot service deploy telephony-network-service --template=telephony-network-service --kubeconfig ~/.kube/tns-central --namespace turandot --input namespace=turandot