Merge pull request #5146 from werf/docs_cleanup_restructure

docs(cleanup): restructure
werf · Dec 13, 2022 · 9524ec3 · 9524ec3
2 parents e615007 + 0635d77
commit 9524ec3
Show file tree

Hide file tree

Showing 29 changed files with 671 additions and 724 deletions.
diff --git a/docs/_data/breadcrumbs.yml b/docs/_data/breadcrumbs.yml
@@ -31,7 +31,7 @@ en:
     url: usage/deploy/releases/release.html
   /usage/cleanup: &documentation-usage-cleanup
     title: Cleanup
-    url: usage/cleanup/cleanup.html
+    url: usage/cleanup/overview.html
   /usage/integration_with_ci_cd_systems: &documentation-usage-integration_with_ci_cd_systems
     title: Integration with CI/CD systems
     url: usage/integration_with_ci_cd_systems/how_ci_cd_integration_works/general_overview.html

diff --git a/docs/_data/sidebars/_documentation.yml b/docs/_data/sidebars/_documentation.yml
@@ -133,11 +133,14 @@ entries:
 
           - title: Cleanup
             f:
-              - title: Cleanup
-                url: /usage/cleanup/cleanup.html
+              - title: Overview
+                url: /usage/cleanup/overview.html
+
+              - title: Container registry cleanup
+                url: /usage/cleanup/cr_cleanup.html
 
-              - title: Supported container registries
-                url: /usage/cleanup/supported_container_registries.html
+              - title: Automatic host cleanup
+                url: /usage/cleanup/host_cleanup.html
 
           - title: Integration with CI/CD systems
             f:
@@ -345,11 +348,14 @@ entries:
 
           - title: Очистка
             f:
-              - title: Очистка
-                url: /usage/cleanup/cleanup.html
+              - title: Обзор
+                url: /usage/cleanup/overview.html
+
+              - title: Очистка container registry
+                url: /usage/cleanup/cr_cleanup.html
 
-              - title: Поддерживаемые container registries
-                url: /usage/cleanup/supported_container_registries.html
+              - title: Автоматическая очистка хоста
+                url: /usage/cleanup/host_cleanup.html
 
           - title: Интеграция с CI/CD системами
             f:

diff --git a/docs/_data/sidebars/documentation.yml b/docs/_data/sidebars/documentation.yml
@@ -763,11 +763,14 @@ entries:
 
           - title: Cleanup
             f:
-              - title: Cleanup
-                url: /usage/cleanup/cleanup.html
+              - title: Overview
+                url: /usage/cleanup/overview.html
+
+              - title: Container registry cleanup
+                url: /usage/cleanup/cr_cleanup.html
 
-              - title: Supported container registries
-                url: /usage/cleanup/supported_container_registries.html
+              - title: Automatic host cleanup
+                url: /usage/cleanup/host_cleanup.html
 
           - title: Integration with CI/CD systems
             f:
@@ -975,11 +978,14 @@ entries:
 
           - title: Очистка
             f:
-              - title: Очистка
-                url: /usage/cleanup/cleanup.html
+              - title: Обзор
+                url: /usage/cleanup/overview.html
+
+              - title: Очистка container registry
+                url: /usage/cleanup/cr_cleanup.html
 
-              - title: Поддерживаемые container registries
-                url: /usage/cleanup/supported_container_registries.html
+              - title: Автоматическая очистка хоста
+                url: /usage/cleanup/host_cleanup.html
 
           - title: Интеграция с CI/CD системами
             f:

diff --git a/docs/_data/werf_yaml.yml b/docs/_data/werf_yaml.yml
@@ -75,9 +75,6 @@ sections:
         description:
           en: Settings for cleaning up irrelevant images
           ru: Настройка удаления неактульных образов
-        detailsAnchor:
-          en: "#cleanup"
-          ru: "#очистка"
         collapsible: true
         isCollapsedByDefault: false
         directives:
@@ -106,9 +103,6 @@ sections:
             description:
               en: Set of policies to select relevant images using the Git history
               ru: Набор политик для выборки актуальных образов, используя историю Git
-            detailsAnchor:
-              en: "#configuring-cleanup-policies"
-              ru: "#конфигурация-политик-очистки"
             directiveList:
               - name: references
                 description:

diff --git a/docs/pages_en/how_to/how_to_migrate_from_v1_1_to_v1_2.md b/docs/pages_en/how_to/how_to_migrate_from_v1_1_to_v1_2.md
@@ -93,7 +93,7 @@ Next, follow these steps:
 
 ### 6. Cleanup by Git history
 
-Remove `--git-history-based-cleanup-v1.2` option for a cleanup. werf always uses git-history cleanup in the v1.2. More info [in the changelog]({{ "/guides/changelog.html#cleanup" | true_relative_url }}) and [in the cleanup article]({{ "/usage/cleanup/cleanup.html" | true_relative_url }}).
+Remove `--git-history-based-cleanup-v1.2` option for a cleanup. werf always uses git-history cleanup in the v1.2. More info [in the changelog]({{ "/guides/changelog.html#cleanup" | true_relative_url }}) and [in the cleanup article]({{ "/usage/cleanup/cr_cleanup.html" | true_relative_url }}).
 
 ### 7. Define environment variables in `werf-giterminism.yaml`
 

diff --git a/docs/pages_en/how_to/integration_with_ci_cd_systems/github_actions/workflows.md b/docs/pages_en/how_to/integration_with_ci_cd_systems/github_actions/workflows.md
@@ -376,7 +376,7 @@ Most cleaning policies in werf are based on git primitives (commit, branch, and
   run: git fetch --prune --unshallow
 ```
 
-werf has an efficient built-in cleanup mechanism to avoid overflowing the container registry and the disk space on the building node with outdated and unused images. You can learn more about the werf's cleanup functionality [here]({{ "usage/cleanup/cleanup.html" | true_relative_url }}).
+werf has an efficient built-in cleanup mechanism to avoid overflowing the container registry and the disk space on the building node with outdated and unused images. You can learn more about the werf's cleanup functionality [here]({{ "usage/cleanup/cr_cleanup.html" | true_relative_url }}).
 
 ## Complete set of configurations for ready-made workflows
 

diff --git a/docs/pages_en/how_to/integration_with_ci_cd_systems/gitlab_ci_cd/workflows.md b/docs/pages_en/how_to/integration_with_ci_cd_systems/gitlab_ci_cd/workflows.md
@@ -122,7 +122,7 @@ In the simplest case, if an integrated container registry is used, then the auth
 
 As a result of the `werf ci-env` command, a temporary docker config is created. It is used by all the commands in the shell session (including docker). It means that parallel tasks do not overlap when using docker, and the temporary token is preserved (not overwritten) in the configuration.
 
-If you need to authorize using the custom credentials, the `docker login` command must be executed after invoking `werf ci-env` (this [article]({{ "/usage/cleanup/supported_container_registries.html#authorization" | true_relative_url }}) sheds more light on authorization).
+If you need to authorize using the custom credentials, the `docker login` command must be executed after invoking `werf ci-env`.
 
 ## Deploying an application
 
@@ -480,7 +480,7 @@ Cleanup:
 ```
 {% endraw %}
 
-werf has an efficient built-in cleanup mechanism to avoid overflowing the container registry and the building node with outdated and unused images. You can learn more about the werf's cleanup functionality [here]({{ "usage/cleanup/cleanup.html" | true_relative_url }}).
+werf has an efficient built-in cleanup mechanism to avoid overflowing the container registry and the building node with outdated and unused images. You can learn more about the werf's cleanup functionality [here]({{ "usage/cleanup/cr_cleanup.html" | true_relative_url }}).
 
 To use cleanup, you have to create a `Personal Access Token` in GitLab with the necessary permissions. This token will be used for authorization in the container registry before cleanup.
 

diff --git a/docs/pages_en/index.md b/docs/pages_en/index.md
@@ -83,11 +83,10 @@ editme_button: false
 <div markdown="1">
  - [Giterminism]({{ "usage/project_configuration/giterminism.html" | true_relative_url }}) describes how determinism is implemented with git, what limitations it imposes, and why.
  - [Helm]({{ "usage/deploy/overview.html" | true_relative_url }})** describes the deploy essentials: how to configure werf for deploying to Kubernetes, what helm chart and release is. Here you may find the basics of templating Kubernetes resources, algorithms for using built images defined in your `werf.yaml` file during the deploy process and working with secrets, plus other useful stuff. Read this chapter if you want to learn more about organizing the deploy process with werf.
- - [Cleanup]({{ "usage/cleanup/cleanup.html" | true_relative_url }}) explains werf cleanup concepts and main commands to perform cleaning tasks.
+ - [Cleanup]({{ "usage/cleanup/overview.html" | true_relative_url }}) explains werf cleanup concepts and main commands to perform cleaning tasks.
  - [CI/CD]({{ "how_to/ci_cd_workflow_basics.html" | true_relative_url }}) describes main aspects of organizing CI/CD workflows with werf. Here you will learn how to use werf with GitLab CI/CD, GitHub Actions, or any other CI/CD system.
  - [Building images with stapel]({{ "reference/werf_yaml.html#image-section" | true_relative_url }}) introduces werf's custom builder. It currently implements the distributed building algorithm to enable lightning-fast build pipelines with distributed caching and incremental rebuilds based on the Git history of your application.
  - [Development and debug]({{ "usage/build/building_images_with_stapel/stage_introspection.html" | true_relative_url }}) describes debugging build and deploy processes of your application when something goes wrong and provides instructions for setting up a local development environment.
- - [Supported container registries]({{ "usage/cleanup/supported_container_registries.html" | true_relative_url }}) contains information about the peculiarities of using various container registries.
 </div>
         </div>
     </div>

diff --git a/docs/pages_en/reference/werf_yaml.md b/docs/pages_en/reference/werf_yaml.md
@@ -82,108 +82,6 @@ deploy:
 
 `deploy.namespaceSlug` defines whether to apply or not [slug]({{ "/usage/deploy/releases/naming.html#slugging-kubernetes-namespace" | true_relative_url }}) to generated kubernetes namespace. Default: `true`.
 
-## Cleanup
-
-### Configuring cleanup policies
-
-The cleanup configuration consists of a set of policies called `keepPolicies`. They are used to select relevant images using the git history. Thus, during a [cleanup]({{ "usage/cleanup/cleanup.html#cleaning-up-outdated-data" | true_relative_url }}), __images not meeting the criteria of any policy are deleted__.
-
-Each policy consists of two parts:
-
-- `references` defines a set of references, git tags, or git branches to perform scanning on.
-- `imagesPerReference` defines the limit on the number of images for each reference contained in the set.
-
-Each policy should be linked to some set of git tags (`tag: string || /REGEXP/`) or git branches (`branch: string || /REGEXP/`). You can specify the name/group of a reference using the [Golang's regular expression syntax](https://golang.org/pkg/regexp/syntax/#hdr-Syntax).
-
-```yaml
-tag: v1.1.1
-tag: /^v.*$/
-branch: main
-branch: /^(main|production)$/
-```
-
-> When scanning, werf searches for the provided set of git branches in the origin remote references, but in the configuration, the  `origin/` prefix is omitted in branch names.
-
-You can limit the set of references on the basis of the date when the git tag was created or the activity in the git branch. The `limit` group of parameters allows the user to define flexible and efficient policies for various workflows.
-
-```yaml
-- references:
-    branch: /^features\/.*/
-    limit:
-      last: 10
-      in: 168h
-      operator: And
-``` 
-
-In the example above, werf selects no more than 10 latest branches that have the `features/` prefix in the name and have shown any activity during the last week.
-
-- The `last: int` parameter allows you to select n last references from those defined in the `branch` / `tag`.
-- The `in: duration string` parameter (you can learn more about the syntax in the [docs](https://golang.org/pkg/time/#ParseDuration)) allows you to select git tags that were created during the specified period or git branches that were active during the period. You can also do that for the specific set of `branches` / `tags`.
-- The `operator: And || Or` parameter defines if references should satisfy both conditions or either of them (`And` is set by default).
-
-When scanning references, the number of images is not limited by default. However, you can configure this behavior using the `imagesPerReference` set of parameters:
-
-```yaml
-imagesPerReference:
-  last: int
-  in: duration string
-  operator: And || Or
-```
-
-- The `last: int` parameter defines the number of images to search for each reference. Their amount is unlimited by default (`-1`).
-- The `in: duration string` parameter (you can learn more about the syntax in the [docs](https://golang.org/pkg/time/#ParseDuration)) defines the time frame in which werf searches for images.
-- The `operator: And || Or` parameter defines what images will stay after applying the policy: those that satisfy both conditions or either of them (`And` is set by default).
-
-> In the case of git tags, werf checks the HEAD commit only; the value of `last`>1 does not make any sense and is invalid
-
-When describing a group of policies, you have to move from the general to the particular. In other words, `imagesPerReference` for a specific reference will match the latest policy it falls under:
-
-```yaml
-- references:
-    branch: /.*/
-  imagesPerReference:
-    last: 1
-- references:
-    branch: master
-  imagesPerReference:
-    last: 5
-```
-
-In the above example, the _master_ reference matches both policies. Thus, when scanning the branch, the `last` parameter will equal to 5.
-
-### Default policies
-
-If there are no custom cleanup policies defined in `werf.yaml`, werf uses default policies configured as follows:
-
-```yaml
-cleanup:
-  keepPolicies:
-  - references:
-      tag: /.*/
-      limit:
-        last: 10
-  - references:
-      branch: /.*/
-      limit:
-        last: 10
-        in: 168h
-        operator: And
-    imagesPerReference:
-      last: 2
-      in: 168h
-      operator: And
-  - references:  
-      branch: /^(main|master|staging|production)$/
-    imagesPerReference:
-      last: 10
-``` 
-
-Let us examine each policy individually:
-
-1. Keep an image for the last 10 tags (by date of creation).
-2. Keep no more than two images published over the past week, for no more than 10 branches active over the past week.
-3. Keep the 10 latest images for main, staging, and production branches.
-
 ## Git worktree
 
 werf stapel builder needs a full git history of the project to perform in the most efficient way. Based on this the default behaviour of the werf is to fetch full history for current git clone worktree when needed. This means werf will automatically convert shallow clone to the full one and download all latest branches and tags from origin during cleanup process.