volterra_namespace resource destroy failing

[Gittins]

Issue

The time it takes for the volterra_namespace resource to be destroyed seems to have increased above the default timeout and is now continuously failing.

Since November 2023 I've been unable to remove namespaces through terraform destroy. Prior to this it was working without issue and taking less than 20secs.

Since November, the time to perform the destroy looks to be taking over 30secs, prior to this it was taking less than 20secs (I'm running Terraform through an automation platform so have historical data on the times). The removal of namespaces also looks to be taking at least 30secs through the XC console now too.

Attached is a capture of the output I'm consistently getting when trying to destroy a namespace.

Destroying of other XC resources through Terraform is not experiencing the same timeout issue (sites, vsites, global networks, vk8s, etc.).

Versions

Terraform: 1.3.2
Volterra provider: Tested using 0.11.22 and 0.11.24

Error message

Sanitised TF output message:

volterra_namespace.ns-example: Refreshing state... [id=309e2315-82a8-4b4d-9bbe-171812d0207b]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  - destroy

Terraform will perform the following actions:

  # volterra_namespace.ns-example will be destroyed
  - resource "volterra_namespace" "ns-example" {
      - id          = "309e2315-82a8-4b4d-9bbe-171812d0207b" -> null
      - name        = "f5demo1611" -> null
      - tenant_name = "mytenant-ckvqcqus" -> null
    }

Plan: 0 to add, 0 to change, 1 to destroy.

Do you really want to destroy all resources?
  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.

  Enter a value: yes

volterra_namespace.ns-example: Destroying... [id=309e2315-82a8-4b4d-9bbe-171812d0207b]
volterra_namespace.ns-example: Still destroying... [id=309e2315-82a8-4b4d-9bbe-171812d0207b, 10s elapsed]
volterra_namespace.ns-example: Still destroying... [id=309e2315-82a8-4b4d-9bbe-171812d0207b, 20s elapsed]
╷
│ Error: Invoking CustomAPI RPC: Doing custom RPC using Rest: Custom API RestClient: Post "https://mytenant.console.ves.volterra.io/api/public/namespaces/f5demo1611/cascade_delete": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

tf_namespace_removal_error_sanitised.txt

[abhinavgrover15]

Ack. Checking on it and will get back.

[Gittins]

Workaround

I've identified, what I would consider, a temporary workaround for this issue.

Increasing the volterra provider timeout to 40secs allows enough time for the deletion of a namespace:

E.g.:

provider "volterra" {
  api_cert = var.api_cert
  api_key = var.api_key
  url = var.volt_api_url
  timeout = "40s"
}

The default value of the provider's timeout parameter/option is 20secs.

A test deletion of a namespace shows this update works and displays the time it is now taking to delete a namespace:

$ terraform destroy
volterra_namespace.ns-showcase-servicenow: Refreshing state... [id=309e2315-4cd6-4520-8399-04391491ba85]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  - destroy

Terraform will perform the following actions:

  # volterra_namespace.ns-showcase-servicenow will be destroyed
  - resource "volterra_namespace" "ns-example" {
      - id          = "309e2315-4cd6-4520-8399-04391491ba85" -> null
      - name        = "provider-test-11-24-3" -> null
      - tenant_name = "mytenant-ckvqcqus" -> null
    }

Plan: 0 to add, 0 to change, 1 to destroy.

Do you really want to destroy all resources?
  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.

  Enter a value: yes

volterra_namespace.ns-example: Destroying... [id=309e2315-4cd6-4520-8399-04391491ba85]
volterra_namespace.ns-example: Still destroying... [id=309e2315-4cd6-4520-8399-04391491ba85, 10s elapsed]
volterra_namespace.ns-example: Still destroying... [id=309e2315-4cd6-4520-8399-04391491ba85, 20s elapsed]
volterra_namespace.ns-example: Destruction complete after 25s

As can be seen in the above output, the time to delete a namespace is now well above the default 20secs timeout value, in this instance 25secs.

I would suggest that either the provider's default timeout is increased (easy option) or individual timeouts per resource are introduced and set at an appropriate rate.

Version info

The following versions were used in this workaround testing:

• terraform: 1.3.2
• volterra provider: 0.11.24