Skip to content

Releases: kubernetes-sigs/kind

v0.6.0

18 Nov 17:33
@BenTheElder BenTheElder
v0.6.0
5c6fdf0
Compare
Choose a tag to compare
v0.6.0

v0.6.0 brings major internal rework and some important breaking changes that we hope will make kind easier to use.

It is also worth noting that kind is now used for blocking Kubernetes pull request testing, in addition to already being used for release signal. This is a major milestone for the project.

Breaking Changes

  • KUBECONFIG is now automatically merged in the same style as kops, minikube, GKE, etc. Please see #1060 for a detailed migration guide.

    • You do not need to and should not do export KUBECONFIG="$(kind get kubeconfig-path)". This command is deprecated and will be removed in a future release.
    • kind will set your current-context when creating a cluster
    • You can configure using a different file by either setting the KUBECONFIG environment variable before creating a cluster, or by setting the --kubeconfig flag on cluster creation

  • Config files are now parsed in strict mode, which should help catch invalid config files

  • kind is moving all external (labels, config) namespaced objects into x-k8s.io to comply with the guidelines for API review. For this release kind will support and use both labels without breaking anything, however in a future release we will only use the new x-k8s.io labels. This note is primarily for @danderson 🙃

  • The default node image is a Kubernetes v1.16.3 image kindest/node:v1.16.3@sha256:bced4bc71380b59873ea3917afe9fb35b00e174d22f50c7cab9188eac2b0fb88

  • Node images built with kind v0.6.0 have many improvements and require kind v0.5.0+, images built with v0.4.0 / v0.3.0 should continue to work with v0.6.0 however.

New Features

  • There is a new kind.x-k8s.io/v1alpha4 config format.
    • Support for per-node kubeadm config patches
    • Support for patching the kind nodes's containerd config, which will be used in an upcoming new guide for local registries
  • Automatic KUBECONFIG export and merge to simplify using kind.
    • kind clusters will always have a kind- prefix in kubeconfig file entries and now can be identified by this.
  • kind export kubeconfig to re-export credentials in the way kind create cluster now does.
  • Significant logging rework
    • -q / --quiet flag to silence status messages / error output
    • Logging with -v 1 / --verbosity 1 or greater will result in detailed error output including stacktraces and command output
  • Colored terminal output and slightly better spinners.
    • You can disable both by setting the environment variable TERM to dumb
  • Enhanced kind version output with more details
    • Builds with make build or make install include git commit
    • The simplified version can be obtained with kind version -q
  • Mounts in extraMounts may have relative paths on the host, resolved against the working directory of kind
  • kubeadm config patches no longer need to include metadata.name and may optionally exclude the apiVersion, allowing more flexible matching of patches to config objects
    • The kind binary is now 75% smaller as a result of eliminating dependencies and writing a patch runtime specific to our needs.
  • kindnetd now has improved logging with klog
  • A random salutation is printed at the end of kind create cluster 🙃
  • It should be possible to install kind via chocolatey with choco install kind. We're also hoping to get a homebrew package shortly!

New Node have been Images for kind v0.5.0, please use these exact images (IE like kindest/node:v1.16.3@sha256:bced4bc71380b59873ea3917afe9fb35b00e174d22f50c7cab9188eac2b0fb88) or build your own as we may need to change the image format again in the future 😅

  • v1.16.3 sha256:bced4bc71380b59873ea3917afe9fb35b00e174d22f50c7cab9188eac2b0fb88
  • v1.15.6 sha256:1c8ceac6e6b48ea74cecae732e6ef108bc7864d8eca8d211d6efb58d6566c40a
  • v1.14.9 sha256:00fb7d424076ed07c157eedaa3dd36bc478384c6d7635c5755746f151359320f
  • v1.13.12 sha256:ad1dd06aca2b85601f882ba1df4fdc03d5a57b304652d0e81476580310ba6289
  • v1.12.10 sha256:e93e70143f22856bd652f03da880bfc70902b736750f0a68e5e66d70de236e40
  • v1.11.10 sha256:44e1023d3a42281c69c255958e09264b5ac787c20a7b95caf2d23f8d8f3746f2

The new node images for recent Kubernetes versions should be > 200 MB smaller thanks to a KEP we implemented upstream to allow building without most of the in-tree cloud provider code.

https://github.com/kubernetes/enhancements/blob/master/keps/sig-cloud-provider/20190729-building-without-in-tree-providers.md

Fixes

  • Various kind CLI output was corrected to be to stdout or stderr where appropriate
  • We have robust new containerd nightly build infrastructure and are shipping the lastest upstream containerd 1.3 + backports in all node images
  • Removed unnecessary dependencies and upgraded all dependencies
  • Fixed building Kubernetes master branch with Bazel
  • Upgraded to build with Go 1.13.4
  • Upgraded all dependencies
  • kindnetd now mounts the iptables xtables lock to ensure coordination with kube-proxy
  • kind will attempt to recreate the original cgroup environment inside the node, this eliminates some error output from kubelet
  • fixed a race condition in command execution, eliminating a source of flakiness of some kind commands
  • kind get kubeconfig does not depend on an existing config file on the host
  • the product_uuid is now faked to be different per-node
  • switched to GA storageclass annotation on the default storage class
    • removed unnecessary addon manager annotation
  • kindnetd is now more robust with backoff on failures
  • kind is now relatively decoupled from docker internally for node management
  • deflaked our cross build and fixed running it on macOS

Contributors

Thanks again to everyone who contributed to this release! ❤️

Users whose commits are in this release (alphabetically by user name)

Assets 8

v0.5.1

21 Aug 20:53
@BenTheElder BenTheElder
v0.5.1
3907329
Compare
Choose a tag to compare
v0.5.1 Pre-release
Pre-release

This release is a minor patch to upgrade kustomize to v3.1.1-0.20190821175718-4b67a6de1296, fixing builds for Windows. This release also contains fixes to our release tooling & CI to ensure we don't regress on this.

Otherwise, please see the release notes for v0.5.0.

See: #792 for more details on what went wrong and how we fixed it.

Assets 8

v0.5.0

20 Aug 22:05
@BenTheElder BenTheElder
v0.5.0
ed9df20
Compare
Choose a tag to compare
v0.5.0 Pre-release
Pre-release

Please see v0.5.1 for a small update with Windows fixes (!)

v0.5.0 primarily brings bug fixes and documentation + code quality improvements

Breaking Changes

  • The default node image is a Kubernetes v1.15.3 image kindest/node:v1.15.3@sha256:27e388752544890482a86b90d8ac50fcfa63a2e8656a96ec5337b902ec8e5157
  • Node images built with kind v0.5.0 will require kind v0.5.0+, images built with v0.4.0 / v0.3.0 should continue to work with v0.5.0 however.

New Features

  • Port forwards now support UDP and SCTP
  • We now have documentation for using kind with WSL2 (shoutout to @PatrickLang)
  • We now have a resources page where you can find talks, guides, and examples
  • Limited arm32 support
  • Nodes perform some more startup logging to the container TTY, which should help with debugging
  • Building node images should be slightly faster due to streamlined image pre-loading

New Node have been Images for kind v0.5.0, please use these exact images or build your own as we may need to change the image format again in the future 😅

  • kindest/node:v1.15.3@sha256:27e388752544890482a86b90d8ac50fcfa63a2e8656a96ec5337b902ec8e5157
  • kindest/node:v1.14.6@sha256:464a43f5cf6ad442f100b0ca881a3acae37af069d5f96849c1d06ced2870888d
  • kindest/node:v1.13.10@sha256:2f5f882a6d0527a2284d29042f3a6a07402e1699d792d0d5a9b9a48ef155fa2a
  • kindest/node:v1.12.10@sha256:e43003c6714cc5a9ba7cf1137df3a3b52ada5c3f2c77f8c94a4d73c82b64f6f3
  • kindest/node:v1.11.10@sha256:bb22258625199ba5e47fb17a8a8a7601e536cd03456b42c1ee32672302b1f909

Fixes

  • Worked around Kubernetes build changes pending in v1.16, kind again supports building Kubernetes at HEAD
  • kind load docker-image now correctly avoids loading images that are already loaded ⚠️
  • More Node runtime written storage is now on volume(s), this should fix some bugs (#771) and improve performance
  • New node images will attempt to work around missing /dev/kmsg causing kubelet to not start
  • / is rshared in new node (images) for mount propagation (more mounts should be shared now)
  • LimitNOFILE in the containerd service now matches the updated upstream value (1048576) in new node images
  • All node level services should be passed proxy environment variables by default now with new node images
  • Upgraded to build with Go 1.12.9
  • Upgraded Kustomize to v3.1.0
  • Bugs in the documentation for using private registries were corrected
  • The kind CLI's own build scripts support proxy environment variables properly
  • We've improved / clarified the roadmap a bit

Contributors

Thanks again to everyone who contributed to this release! 😄

Users whose commits are in this release (alphabetically by user name)

Assets 7

v0.4.0

26 Jun 00:48
@BenTheElder BenTheElder
v0.4.0
08872cf
Compare
Choose a tag to compare
v0.4.0 Pre-release
Pre-release

v0.4.0 brings improved networking features in particular, including initial IPv6 support and additional node port forwards. It also continues to improve speed and reliability (hopefully! 🙃)

Breaking Changes

  • The default node image is a Kubernetes v1.15.0 image kindest/node:v1.15.0@sha256:b4d092fd2b507843dd096fe6c85d06a27a0cbd740a0b32a880fe61aba24bb478
  • The deprecated kind.sigs.k8s.io/v1alpha2 version of config was removed, please switch to kind.sigs.k8s.io/v1alpha3
  • kind build node-image --type=apt was removed. Please use kind build node-image or kind build node-image --type=bazel instead, or one of the pre-built images. A future release will re-work these commands and add support for building from upstream release tarballs.

New Features

  • Additional node port forwards may be configured with the nodes[].extraPortMappings of Cluster configuration in kind.sigs.k8s.io/v1alpha3. This can be used to access workloads more easily from the host. Expect guides using this soon!
  • Limited IPv6 support. The networking.ipFamily may be set to ipv6 in kind.sigs.k8s.io/v1alpha3 Cluster configuration to create an ipv6 enabled cluster on Linux. We have set up continuous Kubernetes conformance testing with IPv6 enabled and are working to fix the tests
  • A warning is emitted if the chosen cluster name is too long and likely to cause the generated node names to be too long.
  • make install now supports overriding INSTALL to specify an alternate tool to install and ensures the output directory
  • Support / documentation for using kind on WSL2.
  • The kind load ... sub-commands now avoid loading images that are already present
  • kubeadm v1beta2 config is now use opportunistically for recent enough Kubernetes versions
  • Reduced startup time for single-node clusters in particular

New Node have been Images for kind v0.4.0, please use these exact images or build your own as we may need to change the image format again in the future 😅

  • kindest/node:v1.15.0@sha256:b4d092fd2b507843dd096fe6c85d06a27a0cbd740a0b32a880fe61aba24bb478
  • kindest/node:v1.14.3@sha256:583166c121482848cd6509fbac525dd62d503c52a84ff45c338ee7e8b5cfe114
  • kindest/node:v1.13.7@sha256:f3f1cfc2318d1eb88d91253a9c5fa45f6e9121b6b1e65aea6c7ef59f1549aaaf
  • kindest/node:v1.12.9@sha256:bcb79eb3cd6550c1ba9584ce57c832dcd6e442913678d2785307a7ad9addc029
  • kindest/node:v1.11.10@sha256:176845d919899daef63d0dbd1cf62f79902c38b8d2a86e5fa041e491ab795d33

Fixes

  • The generated KUBECONFIG file references the API Server by listen address (IP) instead of localhost (domain), which is more correct and should work on systems without a valid localhost entry (!)
  • The new kind images should work on ipv4 only hosts with ipv6 fully disabled
  • The node subnet is properly added included in NO_PROXY on the nodes when proxy settings are detected
  • Clusters with multiple control plane nodes now have proper healthchecks of the API servers in the external loadbalancer
  • Minor cleanup and typo fixes

Contributors

Thanks again to everyone who committed to this release! You all are the best! ❤️

Alphabetically by user name:

Assets 7

v0.3.0

17 May 04:46
@BenTheElder BenTheElder
v0.3.0
838f946
Compare
Choose a tag to compare
v0.3.0 Pre-release
Pre-release

This release focused on improving the speed and reliability of cluster creation,
networking, and bug fixes. A number of breaking changes were required, but Nodes
should be simpler, smaller, faster, and more reliable.

Breaking Changes

IMPORTANT: Breaking Image Changes

kind v0.3+ requires new node images versus v0.2.1 and earlier. A number of
important internal improvements were made that required changing how cluster
bootup is performed. The following images we have previously published are
incompatible with kind v0.3+:

  • kindest/node:v1.11.3
  • kindest/node:v1.12.2
  • kindest/node:v1.12.3
  • kindest/node:v1.13.2
  • kindest/node:v1.13.4
  • kindest/node:v1.14.0

The following images require unreleased versions of kind between v0.3.0 and v0.2.1,
please do not use them:

  • kindest/node:v1.14.1

For 0.3+ we are pushing the following images:

  • kindest/nodev1.14.2@sha256:33539d830a6cf20e3e0a75d0c46a4e94730d78c7375435e6b49833d81448c319
  • kindest/node:v1.13.6@sha256:9e07014fb48c746deb98ec8aafd58c3918622eca6063e643c6e6d86c86e170b4
  • kindest/node:v1.12.8@sha256:cc6e1a928a85c14b52e32ea97a198393fb68097f14c4d4c454a8a3bc1d8d486c
  • kindest/node:v1.11.10@sha256:abd0275ead5ddfd477b7bc491f71957d7dd75408a346834ffe8a9bee5fbdc15b

We hope not to make further breaking changes to the images, but strongly
recommend pinning images by sha256, the default image is pinned to a particular sha.
This can be done with --image=kindest/node:v1.X.Y@sha256:hash. For example:
kind create cluster --image=kindest/node:v1.12.8@sha256:cc6e1a928a85c14b52e32ea97a198393fb68097f14c4d4c454a8a3bc1d8d486c

  • Default clusters will now be Kubernetes v1.14.2, the default node image is
    now kindest/nodev1.14.2@sha256:33539d830a6cf20e3e0a75d0c46a4e94730d78c7375435e6b49833d81448c319

  • Kubernetes on the nodes now uses CRI (containerd) instead of dockershim
    (dockerd) inside the "node" docker containers.
    Please note that details underlying Kubernetes are not guaranteed and may
    change again in the future, however we are still warning about this particular
    change as some advanced users depended on these details anyhow.

  • Similarly, the default CNI / networking setup changed, kind now sets a
    podSubnet and masquerades outbound pod traffic with a new CNI
    setup. This subnet is defaulted but is now configurable and the entire CNI setup
    can be disabled (see features below).

  • Building kind now requires go modules (using any upstream supported go version)
    or using the new Makefile. Pre-built binaries for all platforms are included with
    this release. Please consider using one of the binaries in your CI setups, or
    at least pinning to a tagged release.

New Features

  • The default CNI can be disabled with the Cluster.networking.disableDefaultCNI
    config field, making it easier to experiment with your own CNI.
  • The Pod Subnet is now configurable with Cluster.networking.podSubnet
  • kind completion bash is now supported to source shell completions.
    • WARNING: we are considering renaming this to align better with the [verb] [noun] Kubernetes style CLI structure.
  • Limited PPC64LE support (requires building your own images)
  • The kind command line binary is now built reproducibly
    • building now only requires make / docker on the host as an alternative to go 1.11+ with modules
  • kind now runs a minimal and lightweight CNI setup
  • internal progress on IPv6 support
  • internal progress on cluster reboot support
  • kind is now versioned with go modules

Fixes

  • We've improved auto-detecting the kubernetes source location for kind build node-image
  • Cluster bootup should be even faster and more reliable in constrained environments.
    • simplified startup logic with less waiting
    • Kubernetes images are fully preloaded at build time
    • our networking setup is now somewhat custom, and is simpler and lighter
  • kind export logs should be less likley to flake with large numbers of nodes or
    log files, internally we've combined most of the streams.
  • More improvements to proxy handling, though kind may still not work correctly
    in proxied environments.
  • /sys/class/dmi/id/product_name is faked to kind where applicable, this avoids
    Kubernetes detecting if kind is running on GCE.

Contributors

Thanks again to everyone who committed to this release! ❤️

Alphabetically by user name:

Assets 7

0.2.1

28 Mar 03:58
@BenTheElder BenTheElder
0.2.1
5e29284
Compare
Choose a tag to compare
0.2.1 Pre-release
Pre-release

0.2.1 is a bug fix release

Breaking Changes

NONE

New Features

  • The hostpath provisioner is now enabled by default (#397)

Fixes

  • fix kind build node-image on macOS, previously --type=bazel and --type=docker (the default) did not work properly on not-Linux in 0.2.0 (#413)
  • fix possible panic in failed kind create cluster calls, previously if multiple nodes failed to come up kind could panic (#407)

Contributors

Thanks to everyone who committed to this release! ❤️

Assets 6

0.2.0

20 Mar 19:39
@BenTheElder BenTheElder
0.2.0
a42ca2f
Compare
Choose a tag to compare
0.2.0 Pre-release
Pre-release

Breaking Changes

  • Default clusters will now be Kubernetes v1.13.4, the default node image is now kindest/node:v1.13.4
  • v1alpha1 config is no longer suppported, please upgrade to v1alpha3 (v1alpha2 is also suppported in this release)
  • v1alpha3 is now the default config version
    • The Config type was renamed to Cluster
    • 127.0.0.1 is the default address for binding the API server port to the host, if you wish to use 0.0.0.0 please set this in the networking section of Cluster config in v1alpha3
    • replicas is no longer a field on nodes, given the low node count, please write one entry per node
    • kubeadmConfigPatches and kubeadmConfigPatchesJSON6902 have moved to the Cluster level from nodes (though they were applied cluster-wide previously, and only from the bootstrap control plane node)

New Features

  • Cluster bootup should be faster on Linux in particular, many tasks are now handled in parallel
  • The CNI (weave) is bundled in the node-image, clusters should work without internet access if kindest/node is pulled ahead of time
  • support for multiple-control planes with an external-load balancer, I.E. """HA""" clusters
  • v1alpha3 now supports cluster-wide networking options, which allow for setting the API server listen address and port on the host
  • proxy support is improved
    • http_proxy, https_proxy, and no_proxy are supported instead of their uppercase equivilants if the uppercase values are not set
    • the nodes's docker daemons will be configured with proxy settings from the host (thanks
  • Paths on the host may be bind-mounted to nodes using the new extraMounts field on nodes, to E.G. share a directory with the host and a node
  • We have a new documentation site https://kind.sigs.k8s.io/ 🎉
    • we have a logo now
  • kind is CNCF certified conformant
  • kind should work on ARM, though kubernetes images will need to be built locally for now
  • kind load docker-image and kind load image-archive are now supported for side-loading images into your cluster
  • kubeadm output from init and join will be printed when the kind loglevel is debug
  • kind is now developed with go modules instead of dep
  • Improved usage output for kind create cluster on Windows

Fixes

  • CNI was upgraded to 0.7.5 in the base image
  • Docker was upgraded to 18.06.3 with patched runc
  • kubelet disk eviction and image GC are disabled for kind clusters, this prevents various failures when the host has low disk space. As kind clusters do not run on dedicated hosts, we configure Kubernetes not to reserve disk overhead.
  • Docker image archives are chowned to root:root in the node image, fixing user namespace remapping support
  • No emoji are re-used in CLI output, every action now has a unique emoji

Contributors

Thanks to everyone who committed to this release! ❤️

Alphabetically by user name:

Assets 6

0.1.0 - The second alpha release

16 Jan 05:19
@BenTheElder BenTheElder
0.1.0
2d52d5c
Compare
Choose a tag to compare
0.1.0 - The second alpha release Pre-release
Pre-release

This release contains many exciting features! Amongst other things we now have support for multi-node clusters thanks to @fabriziopandini, and fixes have been made for kind on Windows.

UPDATE 2019-02-11

Due to CVE-2019-5736 ("the runc escape")

kindest/node:v1.13.2@sha256:d070e091e7c0e515a37d0834ea72828b2338acdc4bc7a13cbb84274fb14e5e83
or:
kindest/node:v1.13.3@sha256:d1af504f20f3450ccb7aed63b67ec61c156f9ed3e8b0d973b3dee3c95991753c

Should be used instead of the default image in this release. Example:

kind create cluster --image=kindest/node:v1.13.3@sha256:d1af504f20f3450ccb7aed63b67ec61c156f9ed3e8b0d973b3dee3c95991753c

In the future we are looking at not pinning to an exact image by default, so that minimal fixes can be silently pushed.
We are additionally hoping to adopt rootless containers once support lands in Kubernetes.

Breaking changes since 0.1.0

  • Default clusters will now be Kubernetes v1.13.2, and the default node image is now kindest/node:v1.13.2@sha256:e14edfa4950e009fe560499c9db6e89daae8bd18bcb372caca6d321a86c52cda
  • nodeLifecycleHooks are no longer supported in the config, we believe this feature was unused and will revisit it later, if you need this, please file an issue with your use case!
  • Deleting a cluster now attempts to delete the cluster specific KUBECONFIG file on the host (thanks @blakestoddard!)

New Features

  • The current kind Config is now at kind.sigs.k8s.io/v1alpha2, with support for multi-node clusters! (thanks @fabriziopandini!)
  • The default image was upgraded to Kubernetes v1.13.2 (see above)
  • kind should now work on hosts where userns remap is enabled in the docker daemon (thanks @vasrem!)
  • Improved documentation (thanks @alejandrox1, @jimangel, @tao12345666333, @k8k, @timoreimann, @chuckha, @superbrothers, @benmoss!)
  • kubeadm's config is now internally v1beta1 when new enough Kubernetes images are used, this should improve stable support for future Kubernetes versions. Additionally all config objects are populated, so they may be patched via patches in kind's config.
  • When removing a KUBECONFIG file, we attempt to detect if the environment variable is set and warn about unsetting it (thanks @blakestoddard, @manparvesh!)
  • The base image now contains conntrack, in order to support installing recent kubeadm packages. by default it is now kindest/base:v20181203-d055041
  • kind build node-image now supports a --kube-root flag to explicitly specify the path to Kubernetes's source code, if unset kind will continue to auto-detect this via go's build tooling. (thanks @inercia!)

Fixes

  • On Windows the KUBECONFIG files should now be placed in the default directory correctly (we now use the homedir util from client-go)
  • On Windows patching kubeadm config should work correctly
  • Recent kubeadm debian packages should be installable in the base image after adding conntrack (see above)

Committers to this release are:

Thank you for your contributions!
Additionally, special thanks to @neolit123, @krzyzacy, @munnerz, and @amwat for PR reviews!

Assets 5

0.0.1 - The first alpha test-release

29 Nov 00:40
@BenTheElder BenTheElder
0.0.1
085ba36
Compare
Choose a tag to compare
0.0.1 - The first alpha test-release Pre-release
Pre-release

UPDATE 2019-02-11

Due to CVE-2019-5736 ("the runc escape")

kindest/node:v1.12.5@sha256:2cc6e8153dbe32db0b962cda060e193615951ae8d2a0b808535726a9b6b5e583
or
kindest/node:v1.13.2@sha256:d070e091e7c0e515a37d0834ea72828b2338acdc4bc7a13cbb84274fb14e5e83
or:
kindest/node:v1.13.3@sha256:d1af504f20f3450ccb7aed63b67ec61c156f9ed3e8b0d973b3dee3c95991753c

Should be used instead of the default image in this release. Example:

kind create cluster --image=kindest/node:v1.13.3@sha256:d1af504f20f3450ccb7aed63b67ec61c156f9ed3e8b0d973b3dee3c95991753c

In the future we are looking at not pinning to an exact image by default, so that minimal fixes can be silently pushed.
We are additionally hoping to adopt rootless containers once support lands in Kubernetes.

Please consider upgrading to kind 0.1.0 or newer as well.

UPDATE: Kubernetes v1.12.2 is affected by CVE-2018-1002105; when using kind create cluster please set the image to a non-default image via the --image flag. The following Kubernetes v1.12.3 image should be suitable: kindest/node:v1.12.3@sha256:f0ecb1066697d9417365ca58410132e512ce2010763470bb28c1e8f7fef55464

A patch release will be made to upgrade this default, see #180.

kind is still alpha-grade software, and as such breaking changes will be made to future releases.

This release allows early users to pin to these pre-built binaries and avoid these changes until they intend to upgrade, rather than installing from HEAD with go get ...

NOTE: the default kind create cluster pins to a specific node image in order to avoid breakage there.

This image is: kindest/node:v1.12.2@sha256:6ac1dc1750fc0efd13d4e294115f9012a21282957e4380a5535bd32154193d4d on the current official registry.

Contributors to this release are:

Thank you for your contributions!

Assets 4