karmada v1.5.2 release

Changes since v1.5.1

Bug Fixes

• karmada-controller-manager: Fixed Applied condition of ResourceBinding is always true issue. (#3723, @jwcesign)
• karmada-controller-manager: Fixed the panic issue in case of the grade number of resourceModel is less than the number of resources. (#3610, @sunbinnnnn)
• karmada-scheduler: Fixed the issue that empty deployment can still be propagated to member clusters even when --enableEmptyWorkloadPropagation flag is false. (#3641, @chaunceyjiang)

karmada v1.4.4 release

Changes since v1.4.3

Bug Fixes

• karmada-scheduler: Fixed the issue that empty deployment can still be propagated to member clusters even when --enableEmptyWorkloadPropagation flag is false. (#3642, @chaunceyjiang)
• karmada-controller-manager: Fixed the panic issue in case of the grade number of resourceModel is less than the number of resources. (#3608, @sunbinnnnn)

karmada v1.7.0-alpha.2

Merge pull request #3729 from jwcesign/cronhpa-imp

feat: Support CronFederatedHPA

karmada v1.3.6 release

Changes since v1.3.5

Bug Fixes

• karmada-controller-manager: Fixed the panic issue in case of the grade number of resourceModel is less than the number of resources. (#3607, @sunbinnnnn)

karmada v1.7.0-alpha.1

Merge pull request #3496 from chaunceyjiang/zero_replicas

Fix zero deployment can still propagate to member clusters when EnableEmptyWorkloadPropagation flag is false

karmada v1.6.0 release

What's New

FederatedHPA

Introduced FederatedHPA API to address the requirements that scale workloads across clusters.
FederatedHPA works similarly to HPA in a single cluster. Karmada aggregates metrics from multiple clusters through the karmada-metrics-adaptor component and then scales Pod replicas. The scaled replicas will be distributed to multiple clusters according to the declaration of PropagationPolicy or ClusterPropagationPolicy.

See FederatedHPA Proposal for more details.

(Feature contributor: @chaunceyjiang @jwcesign @Poor12)

Application Failover

In a multi-cluster scenario, the fault may come from the cluster as a whole, or it may be because the application cannot adapt to a certain cluster. Users are now able to declare strategies of application failover about when and how to migrate the unhealthy application in PropagationPolicy and ClusterPropagationPolicy. Karmada will automatically migrate unhealthy applications to other available clusters to improve their availability.

See application-level failover for more details.
(Feature contributor: @Poor12, @RainbowMango)

Karmada Operator

Karmada provides CLI tools and Helm Charts for installation and deployment in previous releases. In this release, the Karmada operator is provided as another declarative deployment method.

The Karmada operator is a method for installing, upgrading, and deleting Karmada instances. It builds upon the basic Karmada resource and controller concepts and provides convenience to centrally manage the entire lifecycle of Karmada instances in a global cluster. With the operator, users can extend Karmada with custom resources (CRs) to manage their instances not only in local clusters but also in remote clusters.

See quick start for more details.
(Feature contributor: @calvin0327, @lonelyCZ, @Poor12)

Third-party Resource Interpreter

Karmada's Resource Interpreter Framework is designed for interpreting resource structure. It consists of built-in and customized interpreters. Karmada has bundled the following open-sourced resources so that users can save the effort to customize them, including Argo Workflow, Flux CD, Kyverno, and OpenKruise. They have been verified by the community.

(Feature contributor: @yike12, @chaunceyjiang, @Poor12)

Other Notable Changes

API Changes

• API change: The length of AffinityName in PropagationPolicy now is restricted to [1, 32], and must be a qualified name. (#3442, @chaunceyjiang)
• API change: Introduced short name wk for resource Work. (#3468, @yanfeng1992)

Bug Fixes

• karmada-webhook: Introduced validation to ensure the .spec.placement.spreadConstraints.maxGroups/minGroups in PropagationPolicy is declared with a reasonable value. (#3232, @whitewindmills )
• karmada-webhook: Validated the predicate path for imageOverride. (#3397, @chaunceyjiang)
• karmada-webhook: Added the missing federatedresourcequota validation config. (#3523, @chaunceyjiang)
• karmada-controller-manager: Fixed the issue that RB/CRB labels were not merged when syncing new changes. (#3239, @lxtywypc)
• karmada-controller-manager: Fixed the issue that control plane endpointslices cannot be deleted. (#3348, @wenchezhao)
• karmada-controller-manager: Corrected the issue of adding duplicate eviction tasks. (#3456, @jwcesign)
• karmada-controller-manager: Fixed a corner case that when there were tasks in the GracefulEvictionTasks queue, graceful-eviction-controller would not work after restarting karmada-controller-manager. (#3475, @chaunceyjiang)
• karmada-controller-manager: Fixed the panic issue in the case that the grade number of resourceModel is less than the number of resources. (#3591, @sunbinnnnn)
• karmadactl: Resolved the failure to view the options of karmadactl addons enable/disable. (#3298, @Poor12)
• karmada-scheduler: Resolved unexpected re-scheduling due to mutating informer cache issue. (#3393, @whitewindmills)
• karmada-scheduler: Fixed the issue of inconsistent Generation and SchedulerObservedGeneration. (#3455, @Poor12)
• karmada-search: Fixed the paging list issue in karmada search proxy in large-scale member clusters. (#3402, @ikaven1024)
• karmada-search: Fixed a panic in ResourceRegistry controller caused by receiving DeletedFinalStateUnknown object from the cache. (#3478, @xigang)
• karmada-search: Fixed contecnt-type header issue in HTTP responses. (#3505, @callmeoldprince)

Security

None

Features & Enhancements

• karmadactl: Introduced --image-pull-secrets flag to init command to specify the secret. (#3237, @my-git9)
• karmadactl: Introduced --force flag to addons disable command. (#3266, @my-git9)
• karmadactl: Introduced support for running init within a pod. (#3338, @lonelyCZ)
• karmadactl: Introduced --host-cluster-domain flag to command init and addons to specify the host cluster domain. (#3292, @tedli)
• karmadactl: Introduced --private-image-registry flag to addons command to specify image registry. (#3345, @my-git9)
• karmadactl: Introduced --purge-namespace flag for deinit command to skip namespace deletion during uninstallation. (#3326, @my-git9)
• karmadactl: Introduced --auto-create-policy and --policy-name flags for promote command to customize the policy during the promotion. (#3494, @LronDC)
• karmada-aggregated-apiserver: Increased .metadata.generation once the desired state of the Cluster object is changed. (#3241, @XiShanYongYe-Chang)
• karmada-controller-mamager: Provided support for Lua's built-in string function in ResourceInterpreterCustomization. (#3256, @chaunceyjiang)
• karmada-controller-manager: The overriders commandsOverrider and argOverride in OverridePolicy now support Job resources. (#3414, @chaunceyjiang)
• karmada-controller-manager: Allowed setting wildcards for --skippedPropagatingNamespaces flag. (#3373, @chaunceyjiang)
• karmada-controller-manager/karmada-agent: Supported connection to resourceInterpretWebhook without DNS Service. (#2999, @lxtywypc)
• karmada-controller-manager: The --skipped-propagating-namespaces flags now can take regular expressions to represent namespaces and defaults to kube-*. (#3433, @chaunceyjiang)
• karmada-controller-manager: Introduced --concurrent-propagation-policy-syncs/--concurrent-cluster-propagation-policy-syncs flags to specify concurrent syncs for PropagationPolicy and ClusterPropagationPolicy. (#3511, @zach593)
• karmada-search: Introduced unified-auth support for proxy. (#3279, @XiShanYongYe-Chang)
• karmada-search: Returned the actual resource list from search API. (#3312, @tedli )
• karmada-search: Fixed the problem that ResourceVersion base64 encrypted repeatedly when starting multiple informers to watch resources. (#3376, @niuyueyang1996)
• karmada-search: Supported namespace filters in RR for search proxy. (#3527, @ikaven1024)
• karmada-scheduler: Optimized the region selection algorithm. (#3259, @whitewindmills)
• karmada-scheduler: Introduced clusterEviction plugin to skip the clusters that are in the process of eviction. (#3469)
• karmada-webhook: Inroduced validation for MultiClusterIngress objects. (#3516, @XiShanYongYe-Chang)

Other

Dependencies

• Karmada is now built with Kubernetes v1.26.2 dependencies. (fix CVE-2022-41723) (#3252, @RainbowMango)
• Karmada (v1.6) is now built with Go 1.20.4. (#3565, @RainbowMango)

Helm Chart

• chart: Introduced controllers config for karmada-controller-manager. (#3240, @Poor12)
• Users can specify com...

karmada v1.5.1 release

Changes since v1.5.0

Bug Fixes

• karmada-search: Fixed the problem that ResourceVersion base64 encrypted repeatedly when starting multiple informers to watch resource. (#3387, @niuyueyang1996)
• karmada-search: Fixed paging list in karmada search proxy in large-scale member clusters issue. (#3449, @ikaven1024)
• karmada-search: Fixed contecnt-type header issue in HTTP response. (#3513, @callmeoldprince)
• karmada-controller-mamager: Fixed Lua's built-in string function can not be used issue in ResourceInterpreterCustomization. (#3282, @chaunceyjiang)
• karmada-controller-manager: Fixed the control plane endpointslices cannot be deleted issue. (#3354, @wenchezhao)
• karmada-controller-manager: Fixed a corner case that when there are tasks in the GracefulEvictionTasks queue, graceful-eviction-controller will not work after restarting karmada-controller-manager. (#3490, @chaunceyjiang)
• karmada-scheduler: Fixed unexpected re-scheduling due to mutating informer cache issue. (#3428, @whitewindmills)
• karmada-scheduler: Fixed the issue of inconsistent Generation and SchedulerObservedGeneration. (#3477, @Poor12)
• karmadactl: Fixed unable to view the options of karmadactl addons enable/disable issue. (#3305, @lonelyCZ)

karmada v1.4.3 release

Changes since v1.4.2

Bug Fixes

• karmada-search: support pod subresource (attach, exec, port-forward) through global proxy. (#3100, @ikaven1024)
• karmada-search: Fixed the problem that ResourceVersion base64 encrypted repeatedly when starting multiple informers to watch resource. (#3388, @niuyueyang1996)
• karmada-search: Fixed paging list in karmada search proxy in large-scale member clusters issue. (#3450, @ikaven1024)
• karmada-search: Fixed contecnt-type header issue in HTTP response. (#3514, @callmeoldprince)
• karmada-controller-manager: Fixed the issue that RBAC resources whose name contains uppercase characters can not be propagated. (#3215, @whitewindmills)
• karmada-controller-mamager: Fixed Lua's built-in string function can not be used issue in ResourceInterpreterCustomization. (#3301, @chaunceyjiang)
• karmada-controller-manager: Fixed the control plane endpointslices cannot be deleted issue. (#3353, @wenchezhao)
• karmadactl: Fixed unable to view the options of karmadactl addons enable/disable issue. (#3306, @lonelyCZ)

karmada v1.3.5 release

Changes since v1.3.4

Bug Fixes

• karmada-search: support pod subresource (attach, exec, port-forward) through global proxy. (#3116, @XiShanYongYe-Chang)
• karmada-search: Fixed the problem that ResourceVersion base64 encrypted repeatedly when starting multiple informers to watch resource. (#3391, @niuyueyang1996)
• karmada-search: Fixed http response add content-type header. (#3515, @callmeoldprince)
• karmada-search: Fiexed paging list in karmada search proxy in large-scale member clusters. (#3451, @ikaven1024)
• karmada-controller-manager: Fixed the issue that RBAC resources whose name contains uppercase characters can not be propagated. (#3214, @whitewindmills)
• karmada-controller-manager: Fixed the problem that ServiceAccount can be interpreted as a dependency, but may not be delivered. (#3222, @whitewindmills)
• karmadactl: Fixed unable to view the options of karmadactl addons enable/disable issue. (#3307, @lonelyCZ)

karmada v1.5.0 release

What's New

Multiple Scheduling Groups

Users are now able to declare multiple groups of clusters to both PropagationPolicy and ClusterPropagationPolicy by leveraging the newly introduced ClusterAffinities field. The scheduler will evaluate these groups one by one in the order they appear in the specification until it finds the one that satisfies scheduling restrictions.

This feature allows the Karmada scheduler to first schedule applications to lower-cost clusters or migrate applications from a primary cluster to backup clusters in the case of cluster failure.

See multiple scheduling group proposal for more info.

(Feature contributor: @XiShanYongYe-Chang @RainbowMango)

New Way to Customize Scheduler

The default scheduler is now able to work with any number of third-party customized schedulers. Similar to Kubernetes, the workloads will be scheduled by the default scheduler if the scheduler name is not declared in PropagationPolicy or ClusterPropagationPolicy.

See customize scheduler for more details.

(Feature contributor: @Poor12)

Other Notable Changes

API Changes

• Introduced Affinities to both PropagationPolicy and ClusterPropagationPolicy. (#3105, @RainbowMango)
• Introduced Placement to the ResoureBinding/ClusterResourceBinding API. (#2702, @Poor12)
• Introduced SchedulerObservedAffinityName to both ResourceBinding and ClusterResourceBinding. (#3163, @RainbowMango)

Bug Fixes

• karmadactl: Fixed the issue that karmada-agent installed by the register command cannot delete works due to lack of permissions. (#2902, @lonelyCZ)
• karmadactl: Fixed the issue that the default ValidatingWebhookConfiguration for resourceinterpreterwebhook was not working. (#2915, @whitewindmills)
• karmadactl: Fixed the issue that the default ValidatingWebhookConfiguration for resourceinterpretercustomizations was not working. (#2916, @chaunceyjiang)
• karmadactl: Fixed the error of resources whose name contains colons failed to be created when using karmadactl apply. (#2919, @Poor12)
• karmadactl init: Granted karmada-agent permissions to access resourceinterpretercustomizations. (#2984, @jwcesign)
• karmada-controller-manager: Generated PolicyRules from given subjects for impersonation deduplicate. (#2911, @yanfeng1992)
• karmada-controller-manager/karmada-agent: Fixed the failure to sync work status due to the informer being accidentally shut down. (#2930, @Poor12)
• karmada-controller-manager/karmada-agent: Fixed misjudgment of deployment and statefuleset health status. (#2928, @Fish-pro)
• karmada-controller-manager: Fixed LabelsOverrider and AnnotationsOverrider failures to add new items in the case of null label/annotation. (#2971, @chaunceyjiang)
• karmada-controller-manager: labelsOverrider/annotationsOverrider supports composed-labels, like testannotation/projectId: . (#3037, @chaunceyjiang)
• karmada-controller-manager: Fixed the issue that RBAC resources whose name contains uppercase characters cannot be propagated. (#3201, @whitewindmills)
• karmada-agent: Check whether the resource exists before creating it. Sometimes the resource is created in advance, giving less privilege to Karmada. (#2988, @jwcesign)
• karmada-scheduler: Fixed a corner case that re-scheduling was skipped in the case that the cluster becomes not fit. (#2912, @jwcesign)
• karmada-search: Filtered out not-ready clusters. (#3010, @yanfeng1992)
• karmada-search: Avoided proxy request block when member clusters were down. (#3027, @ikaven1024)
• karmada-webhook: Validated replicaSchedulingType and replicaDivisionPreference. (#3014, @chaunceyjiang)
• karmada-webhook: Fixed the issue that the InterpretDependency operation cannot be registered. (#3052, @whitewindmills)
• karmada-search: Supported pod subresource (attach, exec, port-forward) through global proxy. (#3098, @ikaven1024)

Security

• golang.org/x/net updates to v0.6.0 to fix CVE-2022-41717. (#3048, @fengshunli)

Features & Enhancements

• karmadactl: Introduced --kube-image-tag flag to the init command to specify the Kubernetes image version. (#2840, @helen-frank)
• karmadactl: The --cluster-context flag of join command now takes current-context by default. (#2956, @helen-frank)
• karmadactl: Added edit mode for interpret commands. (#2831, @ikaven1024)
• karmadactl: Introduced --cert-validity-period for init to make the validity period of cert configurable. (#3156, @lonelyCZ)
• karmada-controller-manager: Now the OverridePolicy and ClusterOverridePolicy will be applied by implicit priority order. The one with the lower priority will be applied before the one with the higher priority. (#2609)
• karmada-controller-manager: Users are now able to apply multiple dependencies interpreter configurations. (#2884, @XiShanYongYe-Chang)
• karmada-controller-manager: Built-in interpreter supports StatefulSets. (#3009, @chaunceyjiang)
• karmada-controller-manager: Retained the labels added to resources by member clusters. (#3088, @chaunceyjiang)
• karmada-controller-manager: Default interpreter supports CronJob aggregated status. (#3129, @chaunceyjiang)
• karmada-controller-manager: Supports PodDisruptionBudget resource in default interpreter. (#2997, @a7i)
• karmada-controller-manager: Support for removing annotations/labels propagated through karmada. (#3099, @chaunceyjiang)
• karmada-webhook: Added validation for policy.spec.placement.orderedClusterAffinities. (#3164, @XiShanYongYe-Chang)
• karmada-webhook: Validated the fieldSelector of overridepolicy. (#3193, @chaunceyjiang)

Other

Dependencies

• Kubernetes images will now be pulled from registry.k8s.io instead of k8s.gcr.io, to be in alignment with current community initiatives. (#2882, @Zhuzhenghao)
• Karmada is now built with Golang 1.19.4. (#2908, @qingwave)
• Karmada is now built with Golang 1.19.5. (#3067, @yanggangtony)
• Karmada is now built with Kubernetes v1.26.1 dependencies. (#3080, @RainbowMango)
• The base image alpine now has been promoted from alpine:3.15.1 to alpine:3.17.1. (#3045, @fengshunli)

Helm Chart

• Fixed helm template missing yaml directive marker to separate API Service resources. (#2963, @a7i)
• Fixed missing karmada-search helm template strategy. (#2994, @a7i)
• Fixed karmada-agent helm template strategy indentation. (#2993, @a7i)
• Chart: karmada-search installation supports specifying an external etcd. (#3120, @my-git9)
• Chart: Supports custom labels variable for etcd. (#3138, @my-git9)

Instrumentation

• Instrumentation: Introduced the pool_get_operation_total, pool_put_operation_total metrics to karmada-controller-manager and karmada-agent. (#2883, @ikaven1024)

Contributors

Thank you to everyone who contributed to this release!

Users whose commits are in this release (alphabetically by username)

• @a7i
• @calvin0327
• @carlory
• @chaunceyjiang
• @fengshunli
• @Fish-pro
• @Garrybest
• @helen-frank
• @ikaven1024
• @jwcesign
• @lonelyCZ
• @maoyangLiu
• @my-git9
• @Poor12
• @qingwave
• @RainbowMango
• @VedRatan
• @Wang-Kai
• @whitewindmills
• @wlp1153468871
• @wongearl
• @XiShanYongYe-Chang
• @yanfeng1992
• @yanggangtony
• @Zhuzhenghao