Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add droplet monitor alert for cloud-init completion #1126

Open
Tythos opened this issue Mar 14, 2024 · 0 comments
Open

Add droplet monitor alert for cloud-init completion #1126

Tythos opened this issue Mar 14, 2024 · 0 comments

Comments

@Tythos
Copy link

Tythos commented Mar 14, 2024

Is your feature request related to a problem? Please describe.

Currently, cloud-init passed via user_data continues for some time beyond when Terraform considers applying a droplet to be "completed. This is reasonable and expected. However, there are a variety of other actions that may require specific cloud-init actions to have completed (package installation, asset fetching from spaces buckets, etc.). Instead of using a provisioner workaround (which is described by Hashicorp as a "last resort" on https://developer.hashicorp.com/terraform/language/resources/provisioners/syntax and has similar issues with signaling execution completion), a more robust path forward would be to leverage the existing monitor alert system.

Describe the solution you'd like

Adding a 'cloud-init-complete' (or similar) feature would let Terraform procedurally construct such alerts as part of the deployment workflow. Currently, this would result in email or slack notifications, which introduces a manual dependency to the deployment path. Optimally, this could be used by on-system scripts or other resource activities (in a way that would currently be checked by "while..." assertion of ps -e | grep cloud results). So, there are two specific modifications I think are relevant:

  1. Addition of a v1/insights/cloud_init_complete to the monitor alert resource types (listed on https://registry.terraform.io/providers/digitalocean/digitalocean/latest/docs/resources/monitor_alert ); under the hood, this can be triggered with a simple check against ps -e | grep cloud results.

  2. An additional (third) alerts block option for appropriate triggers--the most straightforward and simple example I can think of is a path to a shell script to invoke, as such a script could be configured/installed as part of the cloud-init specification. (Why couldn't this just be called from the cloud-init specification itself? Good question. I think such scripts wouldn't focus on continued in-environment execution as much as they would be used to propagate actions to/from other systems. For example, null_resource checks could be used by the Terraform executor to trigger subsequent resource applications.)

Describe alternatives you've considered

Alternatives include:

  • Adding a poll ("while...") loop in an external script

  • Triggering an explicit alert from the end of the cloud-init commands themselves

  • Forgoing cloud-init / user_data entirely and forwarding system configuration responsibilities to Ansible, Chef, Puppet, etc. (though setup of such tools would still be needed)

Additional context

Related to #280 , as such alerts could be used to explicitly trigger other downstream dependencies that would otherwise require a "last resort" provisioner. But more specifically, this is intended to expose downstream dependencies for more automated deployment workflows. Common triggers for post-cloudinit actions include following up with Ansible playbooks or other configuration management assertions. More sophisticated triggers could include Terraform callbacks via null_resource checks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant