docs: external dependencies and resource weight (#4596)

docs: add resource weight and external dependencies

docs/_data/sidebars/_documentation.yml

@@ -70,9 +70,15 @@ entries:
                   - title: Steps
                     url: /advanced/helm/deploy_process/steps.html
 
+                  - title: Deployment order
+                    url: /advanced/helm/deploy_process/deployment_order.html
+
                   - title: Helm hooks
                     url: /advanced/helm/deploy_process/helm_hooks.html
 
+                  - title: External dependencies
+                    url: /advanced/helm/deploy_process/external_dependencies.html
+
                   - title: Annotating and labeling of chart resources
                     url: /advanced/helm/deploy_process/annotating_and_labeling.html
 

docs/_data/sidebars/documentation.yml

@@ -738,9 +738,15 @@ entries:
                   - title: Steps
                     url: /advanced/helm/deploy_process/steps.html
 
+                  - title: Deployment order
+                    url: /advanced/helm/deploy_process/deployment_order.html
+
                   - title: Helm hooks
                     url: /advanced/helm/deploy_process/helm_hooks.html
 
+                  - title: External dependencies
+                    url: /advanced/helm/deploy_process/external_dependencies.html
+
                   - title: Annotating and labeling of chart resources
                     url: /advanced/helm/deploy_process/annotating_and_labeling.html
 

docs/_includes/reference/cli/werf_kubectl_get.md

@@ -14,7 +14,7 @@ Use "kubectl api-resources" for a complete list of supported resources.
 {{ header }} Syntax
 
 ```shell
-werf kubectl get [(-o|--output=)json|yaml|name|go-template|go-template-file|template|templatefile|jsonpath|jsonpath-as-json|jsonpath-file|custom-columns|custom-columns-file|wide] (TYPE[.VERSION][.GROUP] [NAME | -l label] | TYPE[.VERSION][.GROUP]/NAME ...) [flags] [options]
+werf kubectl get [(-o|--output=)json|yaml|name|go-template|go-template-file|template|templatefile|jsonpath|jsonpath-as-json|jsonpath-file|custom-columns-file|custom-columns|wide] (TYPE[.VERSION][.GROUP] [NAME | -l label] | TYPE[.VERSION][.GROUP]/NAME ...) [flags] [options]
 ```
 
 {{ header }} Examples

docs/pages_en/advanced/helm/deploy_process/external_dependencies.md

@@ -0,0 +1,35 @@
+---
+title: External dependencies
+permalink: advanced/helm/deploy_process/external_dependencies.html
+---
+
+To make one release resource dependent on another, you can change its [deployment order]({{ "/advanced/helm/deploy_process/deployment_order.html" | true_relative_url }}) so that the dependent one is deployed after the primary one has been successfully deployed. But what if you want a release resource to depend on a resource that is not part of that release or is not even managed by werf (e.g., created by some operator)?
+
+In this case, the [`<name>.external-dependency.werf.io/resource`]({{ "/reference/deploy_annotations.html#external-dependency-resource" | true_relative_url }}) annotation can help you set the dependency on an external resource. The annotated resource will not be deployed until the defined external dependency is created and ready.
+
+Example:
+```yaml
+kind: Deployment
+metadata:
+  name: app
+  annotations:
+    secret.external-dependency.werf.io/resource: secret/dynamic-vault-secret
+```
+
+In the example above, werf will wait for the `dynamic-vault-secret` to be created before proceeding to deploy the `app` deployment. We assume that `dynamic-vault-secret` is created by the operator from your Vault instance and is not managed by werf.
+
+Let's take a look at another example:
+```yaml
+kind: Deployment
+metadata:
+  name: service1
+  annotations:
+    service2.external-dependency.werf.io/resource: deployment/service2
+    service2.external-dependency.werf.io/namespace: service2-production
+```
+
+Here, werf will not start deploying `service1` until the deployment of `service2` in another namespace has been successfully completed. Note that `service2` can either be deployed as part of another werf release or be managed by some other CI/CD software (i.e., outside of werf's control).
+
+Reference:
+* [`<name>.external-dependency.werf.io/resource`]({{ "/reference/deploy_annotations.html#external-dependency-resource" | true_relative_url }})
+* [`<name>.external-dependency.werf.io/namespace`]({{ "/reference/deploy_annotations.html#external-dependency-namespace" | true_relative_url }})

docs/pages_en/advanced/helm/deploy_process/resource_order.md

@@ -0,0 +1,61 @@
+---
+title: Deployment order
+permalink: advanced/helm/deploy_process/deployment_order.html
+---
+
+By default, resources are applied and tracked simultaneously because they all have the same initial ([werf.io/weight: 0]({{ "/reference/deploy_annotations.html#resource-weight" | true_relative_url }})) (which means that you haven't changed it or deliberately set it to "0"). However, you can change the order in which resources are applied and tracked by setting different weights for them.
+
+Before the deployment phase, werf groups the resources based on their weight, then applies the group with the lowest associated weight and waits until the resources from that group are ready. After all the resources from that group have been successfully deployed, werf proceeds to deploy the resources from the group with the next lowest weight. The process continues until all release resources have been deployed.
+
+> Note that "werf.io/weight" works only for non-Hook resources. For Hooks, use "helm.sh/hook-weight".
+
+Let's look at the example:
+```yaml
+kind: Job
+metadata:
+  name: db-migration
+---
+kind: StatefulSet
+metadata:
+  name: postgres
+---
+kind: Deployment
+metadata:
+  name: app
+---
+kind: Service
+metadata:
+  name: app
+```
+
+All of the above resources will be deployed simultaneously because they all have the same default weight (0). But what if the database must be deployed before migrations, and the application should only start after the migrations are completed? Well, there is a solution to this problem! Try this:
+```yaml
+kind: StatefulSet
+metadata:
+  name: postgres
+  annotations:
+    werf.io/weight: "10"
+---
+kind: Job
+metadata:
+  name: db-migration
+  annotations:
+    werf.io/weight: "20"
+---
+kind: Deployment
+metadata:
+  name: app
+  annotations:
+    werf.io/weight: "30"
+---
+kind: Service
+metadata:
+  name: app
+  annotations:
+    werf.io/weight: "30"
+```
+
+In the above example, werf will first deploy the database and wait for it to become ready, then run migrations and wait for them to complete, and then deploy the application and the related service.
+
+Reference:
+* [`werf.io/weight`]({{ "/reference/deploy_annotations.html#resource-weight" | true_relative_url }})

docs/pages_en/advanced/helm/deploy_process/steps.md

@@ -6,29 +6,13 @@ permalink: advanced/helm/deploy_process/steps.html
 When running the `werf converge` command, werf starts the deployment process that includes the following steps:
 
 1. Rendering chart templates into a single list of Kubernetes resource manifests and validating them.
-2. Running `pre-install` or `pre-upgrade` [hooks]({{ "/advanced/helm/deploy_process/helm_hooks.html" | true_relative_url }}) and tracking each hook until successful or failed termination; printing logs and other information in the process.
-3. Applying changes to Kubernetes resources: creating new, deleting old, updating the existing.
-4. Creating a new release version and saving the current state of resource manifests into this release’s data.
-5. Tracking all release resources until the readiness state is reached; printing logs and other information in the process.
-6. Running `post-install` or `post-upgrade` [hooks]({{ "/advanced/helm/deploy_process/helm_hooks.html" | true_relative_url }}) and tracking each hook until successful or failed termination; printing logs and other information in the process.
-
-**NOTE** werf would delete all newly created resources immediately during the ongoing deploy process if this process fails at any of the steps described above!
-
-When executing helm hooks at the step 2 and 6, werf would track these hooks resources until successful termination. Tracking [can be configured](#resource-tracking) for each hook resource.
+2. Sequentially running `pre-install` or `pre-upgrade` [hooks]({{ "/advanced/helm/deploy_process/helm_hooks.html" | true_relative_url }}) sorted by their weight and tracking each hook to completion while printing its logs in the process.
+3. Grouping non-hook Kubernetes resources by their [weight]({{ "/advanced/helm/deploy_process/deployment_order.html" | true_relative_url }}) and deploying each group sequentially according to its weight: creating/updating/deleting resources and tracking them until they are ready while printing logs in the process.
+4. Running `post-install` or `post-upgrade` [hooks]({{ "/advanced/helm/deploy_process/helm_hooks.html" | true_relative_url }}) the same way as `pre-install` and `pre-upgrade` hooks.
 
 ## Resource tracking
 
-On step 5, werf would track all release resources until each resource reaches the "ready" state. All resources are tracked simultaneously. During tracking, werf aggregates information obtained from all release resources into the single text output in real-time, and periodically prints the so-called status progress table.
-
-werf displays logs of resource Pods until those pods reach the "ready" state. In the case of Job pods, logs are shown until Pods are terminated.
-
-werf uses the [kubedog library](https://github.com/werf/kubedog) to track resources. Currently, tracking is implemented for Deployments, StatefulSets, DaemonSets, and Jobs. Support for tracking Service, Ingress, PVC, and other resources [will be added ](https://github.com/werf/werf/issues/1637).
-
-Tracking behaviour can be configured for each resource using [resource annotations]({{ "/reference/deploy_annotations.html" | true_relative_url }}), which should be set in the chart templates.
-
-## If the deploy failed
-
-In the case of failure during the release process, werf would create a new release having the FAILED state. This state can then be inspected by the user to find the problem and solve it on the next deploy invocation.
+werf uses the [kubedog library](https://github.com/werf/kubedog) to track resources. Tracking behavior can be configured for each resource by setting [resource annotations]({{ "/reference/deploy_annotations.html" | true_relative_url }}) in the chart templates.
 
 ## Multiple Kubernetes clusters
 

docs/pages_en/reference/deploy_annotations.md

@@ -7,6 +7,9 @@ toc: false
 
 This article contains description of annotations which control werf resource operations and tracking of resources during deploy process. Annotations should be configured in the chart templates.
 
+ - [`werf.io/weight`](#resource-weight) — defines the weight of the resource, which will affect the order in which the resources are deployed.
+ - [`<any-name>.external-dependency.werf.io/resource`](#external-dependency-resource) — wait for specified external dependency to be up and running, and only then proceed to deploy the annotated resource.
+ - [`<any-name>.external-dependency.werf.io/namespace`](#external-dependency-namespace) — specify the namespace for the external dependency.
  - [`werf.io/replicas-on-creation`](#replicas-on-creation) — defines number of replicas that should be set only when creating resource initially (useful for HPA).
  - [`werf.io/track-termination-mode`](#track-termination-mode) — defines a condition when werf should stop tracking of the resource.
  - [`werf.io/fail-mode`](#fail-mode) — defines how werf will handle a resource failure condition which occurred after failures threshold has been reached for the resource during deploy process.
@@ -21,6 +24,42 @@ This article contains description of annotations which control werf resource ope
 
 More info about chart templates and other stuff is available in the [helm chapter]({{ "advanced/helm/overview.html" | true_relative_url }}).
 
+## Resource weight
+
+`werf.io/weight: "NUM"`
+
+Example: \
+`werf.io/weight: "10"` \
+`werf.io/weight: "-10"`
+
+- Can be a positive number, a negative number, or a zero. 
+- Value is passed as a string. If omitted, the `weight` is set to 0 by default. 
+- Works for non-Hook resources only. For Hooks, use `helm.sh/hook-weight`, which works almost the same.
+
+This parameter sets the weight of the resources, defining the order in which they are deployed. First, werf groups resources according to their weight and then sequentially deploys them, starting with the group with the lowest weight. In this case, werf will not proceed to deploy the next batch of resources until the previous group has been successfully deployed.
+
+More info: [deployment order]({{ "/advanced/helm/deploy_process/deployment_order.html" | true_relative_url }})
+
+## External dependency resource
+
+`<any-name>.external-dependency.werf.io/resource: type[.version.group]/name`
+
+Example: \
+`secret.external-dependency.werf.io/resource: secret/config` \
+`someapp.external-dependency.werf.io/resource: deployments.v1.apps/app`
+
+Sets the external dependency for the resource. The annotated resource won't be deployed until the external dependency has been created and ready.
+
+More info: [external dependencies]({{ "/advanced/helm/deploy_process/external_dependencies.html" | true_relative_url }})
+
+## External dependency namespace
+
+`<any-name>.external-dependency.werf.io/namespace: name`
+
+Sets the namespace for the external dependency specified by the [external dependency resource](#external-dependency-resource) annotation. The `<any-name>` prefix must be the same as in the annotation of the external dependency resource.
+
+More info: [external dependencies]({{ "/advanced/helm/deploy_process/external_dependencies.html" | true_relative_url }})
+
 ## Replicas on creation
 
 When HPA is active usage of default `spec.replicas` leads to harmful and tricky behaviour, because each time werf chart is being converged through CI/CD process, resource replicas will be reset to the static `spec.replicas` value in the chart templates, even if this value will was already changed in runtime by the HPA.