ConnectionTimeout

[dwizzle204]

Is there anyway to specify connection attempts in the verifier?
It appears like kitchen is trying to login before the VM is accepting ssh connections....
If I don't do the destroy I can ssh into the server using the proper account without issue.

[SSH] connection failed, retrying in 1 seconds (#<Net::SSH::ConnectionTimeout: Net::SSH::ConnectionTimeout>)
[SSH] connection failed, retrying in 1 seconds (#<Errno::ECONNREFUSED: Connection refused - connect(2) for 10.84.4.29:22>)
[SSH] connection failed, retrying in 1 seconds (#<Errno::ECONNREFUSED: Connection refused - connect(2) for 10.84.4.29:22>)
[SSH] connection failed, retrying in 1 seconds (#<Errno::ECONNREFUSED: Connection refused - connect(2) for 10.84.4.29:22>)
$$$$$$ [SSH] connection failed, terminating (#<Errno::ECONNREFUSED: Connection refused - connect(2) for 10.84.4.29:22>)

------Exception-------
Class: Kitchen::ActionFailed
Message: 1 actions failed.

Example kitchen file:

driver:
name: terraform
command_timeout: 600
color: true
root_module_directory: terraform/

provisioner:
name: terraform

platforms:

• name: rhel7
  os_type: linux

transport:
name: ssh
connection_retries: 600
connection_retry_sleep: 10
max_wait_until_ready: 6000

verifier:
name: terraform
systems:
- name: image
backend: ssh
user: packer
key_files:
- packer-ssh-key
hosts_output: host_ip
sudo: true
reporter:
- junit:inspec_output.xml
input_files:
- test/integration/image/attributes.yml # attributes that were previously passed in here, now need to be in a file

suites:

• name: image

[aaron-lane]

Hi @dwizzle204! Thank you for your interest in the project!

If there is a sufficient delay between kitchen converge and kitchen verify, does the connection succeed or does it only succeed when executed manually?

Transport plugins are not currently used by the Kitchen-Terraform verifier plugin as it relies on InSpec to handle connection responsibilities. I think you should be able to use the proxy_command attribute of the system to control connection retry logic, though admittedly the solution is convoluted as it relies on Kitchen config interpolation.

driver:
  name: terraform
  command_timeout: 600
  color: true
  root_module_directory: terraform/

provisioner:
  name: terraform

platforms:
  - name: rhel7
    os_type: linux

verifier:
  name: terraform
  systems:
    - name: image
      backend: ssh
      proxy_command: ssh packer@<%= `terraform output host_ip -state terraform/terraform.tfstate.d/terraform.tfstate` %> -o 'ConnectionAttempts 10' -o 'ConnectTimeout 5' -W %h:%p
      user: packer
      key_files:
        - packer-ssh-key
      hosts_output: host_ip
      sudo: true
      reporter:
        - junit:inspec_output.xml
      input_files:
        - test/integration/image/attributes.yml

suites:
  - name: image

[dwizzle204]

Thanks for the response.

I modified this slight as the order was off

  proxy_command: ssh packer@<%= `terraform output -state terraform/terraform.tfstate.d/terraform.tfstate host_ip` %> -o 'ConnectionAttempts 10' -o 'ConnectTimeout 5' -W %h:%p

But it looks like its trying to populate that info before the test vm is built

• kitchen test --destroy never
  -----> Starting Test Kitchen (v2.3.4)
  The module root could not be found. There is nothing to output.
  -----> Cleaning up any prior instances of
  -----> Destroying ...
  Terraform v0.11.11

-----> Verifying ...
$$$$$$ Running command terraform workspace select kitchen-terraform-rhel7 in directory ***/terraform
$$$$$$ Running command terraform output -json in directory ***/terraform
ssh: Could not resolve hostname : Name or service not known

------Exception-------
Class: Kitchen::ActionFailed
Message: 1 actions failed.
Verify failed on instance . Please see .kitchen/logs/**.log for more details

Please see .kitchen/logs/kitchen.log for more details
Also try running kitchen diagnose --all for configuration

[aaron-lane]

I forgot to mention the caveat that you will need to run kitchen verify as a separate step to ensure that terraform output can resolve successfully. 😊

[aaron-lane]

It's clear to me that this experience is not great. We'll have to see if there's a better way to handle it through InSpec's transport interface.

[dwizzle204]

In the meantime I'm able to get around it by adding this to my terraform

resource "null_resource" "add_delay" {
depends_on = ["google_compute_instance.testing"]
provisioner "local-exec" {
command = "sleep 120"
}
}

[tusca]

a variant on this temporary workaround can be to use netcat, also works with multiple instances

resource "null_resource" "wait_for_instance" {
  depends_on = [SOMETHING]
  count      = N
 
  provisioner "local-exec" {
    command = "while ! nc -z ${INSTANCE[count.index]} 22; do sleep 1 ; done; sleep 1"
  }
}

where you have to replace the SOMETHING, INSTANCE, N with relevant values/references ;-)