Longhorn v1.5.5

Longhorn v1.5.5 Release Notes

This latest version of Longhorn 1.5 introduces several improvements and bug fixes that are intended to improve system quality, resilience, and stability.

The Longhorn team appreciates your contributions and anticipates receiving feedback regarding this release.

Note

For more information about release-related terminology, see Releases.

Installation

Important

Ensure that your cluster is running Kubernetes v1.21 or later before installing Longhorn v1.5.5.

You can install Longhorn using a variety of tools, including Rancher, Kubectl, and Helm. For more information about installation methods and requirements, see Quick Installation in the Longhorn documentation.

Upgrade

Important

Ensure that your cluster is running Kubernetes v1.21 or later before upgrading from Longhorn v1.4.x or v1.5.x (< v1.5.5) to v1.5.5.

Longhorn only allows upgrades from supported versions. For more information about upgrade paths and procedures, see Upgrade in the Longhorn documentation.

Deprecation & Incompatibilities

For information about important changes, including feature incompatibility, deprecation, and removal, see Important Notes in the Longhorn documentation.

Post-Release Known Issues

For information about issues identified after this release, see Release-Known-Issues.

Resolved Issues

Improvements

• [BACKPORT][v1.5.5][IMPROVEMENT] Cannot read/write to block volume when the container is run as non-root 8123 - @PhanLe1010 @chriscchien
• [BACKPORT][v1.5.5][IMPROVEMENT] Do not terminate nfs-ganesha in share-manager pod after failing to access recovery backend 8347 - @derekbit @chriscchien
• [BACKPORT][v1.5.5][IMPROVEMENT] Expose virtual size of qcow2 backing images 8321 - @shuo-wu @chriscchien
• [BACKPORT][v1.5.5][IMPROVEMENT] Improve logging in CSI plugin when mount fails. 8286 - @james-munson @chriscchien
• [BACKPORT][v1.5.5][IMPROVEMENT] Upgrade support bundle kit version to v0.0.36 8161 - @c3y1huang @roger-ryao
• [BACKPORT][v1.5.5][IMPROVEMENT] Improve environment_check script for NFS protocol bug and the host system self diagnosis 7972 - @james-munson @roger-ryao

Bug Fixes

• Security issues in latest longhorn docker images 8372 - @c3y1huang @chriscchien
• [BACKPORT][v1.5.5][BUG] Backup marked as "completed" cannot be restored, gzip: invalid header 8378 - @derekbit @chriscchien
• [BACKPORT][v1.5.5][BUG][v1.6.0-rc1] Failed to run instance-manager in storage network environment 8305 - @yangchiu @ejweber
• [BACKPORT][v1.5.5][BUG] Replica rebuild failed 8257 - @shuo-wu @chriscchien
• [BACKPORT][v1.5.5][BUG] longhorn manager pod fails to start in container-based K3s 7948 - @ChanYiLin @chriscchien
• [BACKPORT][v1.5.5][BUG] persistence.removeSnapshotsDuringFilesystemTrim Helm variable is unreferenced 7951 - @ejweber @roger-ryao
• [BACKPORT][v1.5.5][BUG] Failed to restore a backup to file by the scripts/restore-backup-to-file.sh with a CIFS backup target. 8127 - @mantissahz @roger-ryao
• [BACKPORT][v1.5.5][BUG] Longhorn api-server PUT request rate 8153 - @ejweber @roger-ryao
• [BACKPORT][v1.5.5][BUG] A replica may be incorrectly scheduled to a node with an existing failed replica 8116 - @ejweber @chriscchien
• [BACKPORT][v1.5.5][BUG] potential risk to unmap a negative number 8236 - @Vicente-Cheng @roger-ryao
• [BACKPORT][v1.5.5][BUG] Use config map to update default-replica-count won't apply to default-replica-count.definition.default if the value equal to current default-replica-count.value 8135 - @james-munson @chriscchien
• [BACKPORT][v1.5.5][BUG] LH manager reboots due to the webhook is not ready 8036 - @ChanYiLin @chriscchien
• [BACKPORT][v1.5.5][BUG] Can't use longhorn with Generic ephemeral volumes 8201 - @ejweber @roger-ryao
• [BACKPORT][v1.5.5][BUG] Volume cannot attach because of the leftover non-empty volume.status.PendingNodeID after upgrading Longhorn 7996 - @james-munson @roger-ryao
• [BACKPORT][v1.5.5][BUG] no Pending workload pods for volume xxx to be mounted 8082 - @c3y1huang @roger-ryao

Miscellaneous

• [TASK] update go-iscsi-helper and go-common-lib in v1.5.x 7960 - @ChanYiLin
• [BACKPORT][v1.5.5][REFACTOR] move mount point check function to common lib 8125 - @ChanYiLin @roger-ryao

Contributors

• @ChanYiLin
• @PhanLe1010
• @Vicente-Cheng
• @c3y1huang
• @chriscchien
• @derekbit
• @ejweber
• @innobead
• @james-munson
• @mantissahz
• @roger-ryao
• @shuo-wu
• @yangchiu

Longhorn v1.5.5-rc2

DON'T UPGRADE from/to any RC/Preview/Sprint releases because the operation is not supported.

Resolved Issues in this release

Improvement

• [BACKPORT][v1.5.5][IMPROVEMENT] Cannot read/write to block volume when the container is run as non-root 8123 - @PhanLe1010 @chriscchien
• [BACKPORT][v1.5.5][IMPROVEMENT] Do not terminate nfs-ganesha in share-manager pod after failing to access recovery backend 8347 - @derekbit @chriscchien
• [BACKPORT][v1.5.5][IMPROVEMENT] Expose virtual size of qcow2 backing images 8321 - @shuo-wu @chriscchien
• [BACKPORT][v1.5.5][IMPROVEMENT] Improve logging in CSI plugin when mount fails. 8286 - @james-munson @chriscchien
• [BACKPORT][v1.5.5][IMPROVEMENT] Upgrade support bundle kit version to v0.0.36 8161 - @c3y1huang @roger-ryao
• [BACKPORT][v1.5.5][IMPROVEMENT] Improve environment_check script for NFS protocol bug and the host system self diagnosis 7972 - @james-munson @roger-ryao

Bug

• [BACKPORT][v1.5.5][BUG][v1.6.0-rc1] Failed to run instance-manager in storage network environment 8305 - @yangchiu @ejweber
• [BACKPORT][v1.5.5][BUG] Replica rebuild failed 8257 - @shuo-wu @chriscchien
• [BACKPORT][v1.5.5][BUG] longhorn manager pod fails to start in container-based K3s 7948 - @ChanYiLin @chriscchien
• [BACKPORT][v1.5.5][BUG] persistence.removeSnapshotsDuringFilesystemTrim Helm variable is unreferenced 7951 - @ejweber @roger-ryao
• [BACKPORT][v1.5.5][BUG] Failed to restore a backup to file by the scripts/restore-backup-to-file.sh with a CIFS backup target. 8127 - @mantissahz @roger-ryao
• [BACKPORT][v1.5.5][BUG] Longhorn api-server PUT request rate 8153 - @ejweber @roger-ryao
• [BACKPORT][v1.5.5][BUG] A replica may be incorrectly scheduled to a node with an existing failed replica 8116 - @ejweber @chriscchien
• [BACKPORT][v1.5.5][BUG] potential risk to unmap a negative number 8236 - @Vicente-Cheng @roger-ryao
• [BACKPORT][v1.5.5][BUG] Use config map to update default-replica-count won't apply to default-replica-count.definition.default if the value equal to current default-replica-count.value 8135 - @james-munson @chriscchien
• [BACKPORT][v1.5.5][BUG] LH manager reboots due to the webhook is not ready 8036 - @ChanYiLin @chriscchien
• [BACKPORT][v1.5.5][BUG] Can't use longhorn with Generic ephemeral volumes 8201 - @ejweber @roger-ryao
• [BACKPORT][v1.5.5][BUG] Volume cannot attach because of the leftover non-empty volume.status.PendingNodeID after upgrading Longhorn 7996 - @james-munson @roger-ryao
• [BACKPORT][v1.5.5][BUG] no Pending workload pods for volume xxx to be mounted 8082 - @c3y1huang @roger-ryao

Misc

• [TASK] update go-iscsi-helper and go-common-lib in v1.5.x 7960 - @ChanYiLin
• [BACKPORT][v1.5.5][REFACTOR] move mount point check function to common lib 8125 - @ChanYiLin @roger-ryao

Contributors

• @ChanYiLin
• @PhanLe1010
• @Vicente-Cheng
• @c3y1huang
• @chriscchien
• @derekbit
• @ejweber
• @innobead
• @james-munson
• @mantissahz
• @roger-ryao
• @shuo-wu
• @yangchiu

Longhorn v1.5.5-rc1

DON'T UPGRADE from/to any RC/Preview/Sprint releases because the operation is not supported.

Resolved Issues in this release

Improvement

• [BACKPORT][v1.5.5][IMPROVEMENT] Expose virtual size of qcow2 backing images 8321 - @chriscchien
• [BACKPORT][v1.5.5][IMPROVEMENT] Cannot read/write to block volume when the container is run as non-root 8123 - @PhanLe1010 @chriscchien
• [BACKPORT][v1.5.5][IMPROVEMENT] Improve logging in CSI plugin when mount fails. 8286 - @james-munson @chriscchien
• [BACKPORT][v1.5.5][IMPROVEMENT] Allow users to request backup volume update 7983 -
• [BACKPORT][v1.5.5][IMPROVEMENT] Upgrade support bundle kit version to v0.0.36 8161 - @c3y1huang @roger-ryao
• [BACKPORT][v1.5.5][IMPROVEMENT] Improve environment_check script for NFS protocol bug and the host system self diagnosis 7972 - @james-munson @roger-ryao

Bug

• [BACKPORT][v1.5.5][BUG] longhorn manager pod fails to start in container-based K3s 7948 - @ChanYiLin @chriscchien
• [BACKPORT][v1.5.5][BUG][v1.6.0-rc1] Failed to run instance-manager in storage network environment 8305 - @yangchiu @ejweber
• [BACKPORT][v1.5.5][BUG] RWX volume is hang on Photon OS 8280 - @PhanLe1010
• [BACKPORT][v1.5.5][BUG] persistence.removeSnapshotsDuringFilesystemTrim Helm variable is unreferenced 7951 - @ejweber @roger-ryao
• [BACKPORT][v1.5.5][BUG] Failed to restore a backup to file by the scripts/restore-backup-to-file.sh with a CIFS backup target. 8127 - @mantissahz @roger-ryao
• [BACKPORT][v1.5.5][BUG] Longhorn api-server PUT request rate 8153 - @ejweber @roger-ryao
• [BACKPORT][v1.5.5][BUG] A replica may be incorrectly scheduled to a node with an existing failed replica 8116 - @ejweber @chriscchien
• [BACKPORT][v1.5.5][BUG] potential risk to unmap a negative number 8236 - @Vicente-Cheng @roger-ryao
• [BACKPORT][v1.5.5][BUG] Replica rebuild failed 8257 - @shuo-wu @chriscchien
• [BACKPORT][v1.5.5][BUG] Use config map to update default-replica-count won't apply to default-replica-count.definition.default if the value equal to current default-replica-count.value 8135 - @james-munson @chriscchien
• [BACKPORT][v1.5.5][BUG] LH manager reboots due to the webhook is not ready 8036 - @ChanYiLin @chriscchien
• [BACKPORT][v1.5.5][BUG] Can't use longhorn with Generic ephemeral volumes 8201 - @ejweber @roger-ryao
• [BACKPORT][v1.5.5][BUG] Volume cannot attach because of the leftover non-empty volume.status.PendingNodeID after upgrading Longhorn 7996 - @james-munson @roger-ryao
• [BACKPORT][v1.5.5][BUG] no Pending workload pods for volume xxx to be mounted 8082 - @c3y1huang @roger-ryao
• [BACKPORT][v1.5.5][BUG] Panic during collecting metrics 8102 - @derekbit
• [BACKPORT][v1.5.5][BUG] Missed NodeStageVolume after reboot leads to CreateContainerError 8104 - @ejweber

Misc

• [TASK] update go-iscsi-helper and go-common-lib in v1.5.x 7960 - @ChanYiLin
• [BACKPORT][v1.5.5][REFACTOR] move mount point check function to common lib 8125 - @ChanYiLin @roger-ryao

Contributors

• @ChanYiLin
• @PhanLe1010
• @Vicente-Cheng
• @c3y1huang
• @chriscchien
• @derekbit
• @ejweber
• @innobead
• @james-munson
• @mantissahz
• @roger-ryao
• @shuo-wu
• @yangchiu

Longhorn v1.6.1

Longhorn v1.6.1 Release Notes

Longhorn 1.6.1 introduces several improvements and bug fixes that are intended to improve system quality, resilience, and stability.

The Longhorn team appreciates your contributions and expects to receive feedback regarding this release.

Note

For more information about release-related terminology, see Releases.

Installation

Important

Ensure that your cluster is running Kubernetes v1.21 or later before installing Longhorn v1.6.1.

You can install Longhorn using a variety of tools, including Rancher, Kubectl, and Helm. For more information about installation methods and requirements, see Quick Installation in the Longhorn documentation.

Upgrade

Important

Ensure that your cluster is running Kubernetes v1.21 or later before upgrading from Longhorn v1.5.x or v1.6.x to v1.6.1.

Longhorn only allows upgrades from supported versions. For more information about upgrade paths and procedures, see Upgrade in the Longhorn documentation.

Deprecation & Incompatibilities

For information about important changes, including feature incompatibility, deprecation, and removal, see Important Notes in the Longhorn documentation.

Post-Release Known Issues

For information about issues identified after this release, see Release-Known-Issues.

Resolved Issues

Improvement

• [BACKPORT][v1.6.1][IMPROVEMENT] Add dmsetup and dmcrypt utilities check in environment check script 8234 - @derekbit @chriscchien
• [BACKPORT][v1.6.1][IMPROVEMENT] Upgrade support bundle kit version to v0.0.36 8162 - @c3y1huang @roger-ryao
• [BACKPORT][v1.6.1][IMPROVEMENT] Cannot read/write to block volume when the container is run as non-root 8122 - @PhanLe1010 @chriscchien
• [BACKPORT][v1.6.1][IMPROVEMENT] Improve environment_check script for NFS protocol bug and the host system self diagnosis 7971 - @james-munson @roger-ryao
• [BACKPORT][v1.6.1][IMPROVEMENT] Use HEAD instead of a GET to fetch the Content-Length of an resource via URL 7973 - @votdev @roger-ryao
• [BACKPORT][v1.6.1][IMPROVEMENT] Remove startup probe of CSI driver after liveness probe conn fix ready 7886 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][IMPROVEMENT] Change support-bundle-manager image pull policy to PullIfNotPresent 8000 - @ChanYiLin @roger-ryao

Bug

• [BUG][v1.6.1-rc3] test_backup_lock_creation_during_deletion failed with *.lck type 1 acquisition 8269 - @ChanYiLin @chriscchien
• [BACKPORT][v1.6.1][BUG] Replica rebuild failed 8258 - @shuo-wu @roger-ryao
• [BACKPORT][v1.6.1][BUG] potential risk to unmap a negative number 8237 - @Vicente-Cheng @roger-ryao
• [BACKPORT][v1.6.1][BUG] Can't use longhorn with Generic ephemeral volumes 8213 - @ejweber @chriscchien
• [BUG] [v1.6.1-rc1] v2 volume replica offline rebuilding fail 8187 - @shuo-wu @chriscchien
• [BACKPORT][v1.6.1][BUG] Longhorn api-server PUT request rate 8152 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][BUG] Use config map to update default-replica-count won't apply to default-replica-count.definition.default if the value equal to current default-replica-count.value 8134 - @james-munson @roger-ryao
• [BACKPORT][v1.6.1][BUG] Failed to restore a backup to file by the scripts/restore-backup-to-file.sh with a CIFS backup target. 8128 - @mantissahz @roger-ryao
• [BACKPORT][v1.6.1][BUG] Exporting data from the existing replicas has issues in 1.6.0 8096 - @ChanYiLin @chriscchien
• [BACKPORT][v1.6.1][BUG] longhorn manager pod fails to start in container-based K3s 7847 - @ChanYiLin @khushboo-rancher @chriscchien
• [BACKPORT][v1.6.1][BUG] no Pending workload pods for volume xxx to be mounted 8081 - @c3y1huang @chriscchien
• [BACKPORT][v1.6.1][BUG] Deadlock is possible in v1.6.0 instance manager 7920 - @roger-ryao
• [BACKPORT][v1.6.1][BUG][1.6.0] ENGINE v2 : disk /dev/xxxx is already used by AIO bdev disk-x 8133 - @derekbit @chriscchien
• [BACKPORT][v1.6.1][BUG] A replica may be incorrectly scheduled to a node with an existing failed replica 8044 - @ejweber @chriscchien
• [BACKPORT][v1.6.1][BUG] Deadlock between volume migration and upgrade after Longhorn upgrade 7870 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][BUG] Volume cannot attach because of the leftover non-empty volume.status.PendingNodeID after upgrading Longhorn 7995 - @james-munson @chriscchien
• [BACKPORT][v1.6.1][BUG] Longhorn may keep corrupted salvaged replicas and discard good ones 7885 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][BUG] LH manager reboots due to the webhook is not ready 8037 - @ChanYiLin @chriscchien
• [BACKPORT][v1.6.1][BUG] BackingImage does not download URL correctly in some situation 7987 - @votdev @yangchiu
• [BACKPORT][v1.6.1][BUG] Executing fstrim while rebuilding causes IO errors 7868 - @yangchiu @ejweber
• [BACKPORT][v1.6.1][BUG] The feature of auto remount read only volume not work on a single node cluster. 7846 - @yangchiu @ChanYiLin
• [BACKPORT][v1.6.1][BUG] Missed NodeStageVolume after reboot leads to CreateContainerError 8012 - @ejweber @chriscchien
• [BACKPORT][v1.6.1][BUG] persistence.removeSnapshotsDuringFilesystemTrim Helm variable is unreferenced 7952 - @ejweber @chriscchien
• [BACKPORT][v1.6.1][BUG][v1.5.4-rc4] Test case test_backuptarget_available_during_engine_image_not_ready failed to wait for backup target available 8055 - @c3y1huang @chriscchien
• [BACKPORT][v1.6.1][BUG] Add Snapshot Maximum Count to the Settings 7979 - @FrankYang0529 @roger-ryao
• [BACKPORT][v1.6.1][BUG] Volumes stuck upgrading after 1.5.3 -> 1.6.0 upgrade. 7899 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][BUG] The activated DR volume do not contain the latest data. 7946 - @shuo-wu @roger-ryao
• [BACKPORT][v1.6.1][BUG][v1.5.x] Recurring job fails to create backup when volume detached 8013 - @yangchiu @mantissahz @PhanLe1010 @c3y1huang
• [BACKPORT][v1.6.1][BUG] Create backup failed: failed lock lock-*.lck type 1 acquisition 7875 - @yangchiu @ChanYiLin @chriscchien
• [BUG] Fix errors in questions.yaml 6392 - @james-munson @chriscchien

Misc

• [BACKPORT][v1.6.1][REFACTOR] move mount point check function to common lib 8124 - @ChanYiLin @chriscchien
• [TASK] Fix updating patch digest dependencies (v1.6.x) 8107 - @mantissahz

Contributors

• @ChanYiLin
• @FrankYang0529
• @PhanLe1010
• @Vicente-Cheng
• @c3y1huang
• @chriscchien
• @derekbit
• @ejweber
• @innobead
• @james-munson
• @khushboo-rancher
• @mantissahz
• @roger-ryao
• @shuo-wu
• @votdev
• @yangchiu

Longhorn v1.6.1-rc3

DON'T UPGRADE from/to any RC/Preview/Sprint releases because the operation is not supported.

Resolved Issues in this release

Improvement

• [BACKPORT][v1.6.1][IMPROVEMENT] Add dmsetup and dmcrypt utilities check in environment check script 8234 - @derekbit @chriscchien
• [BACKPORT][v1.6.1][IMPROVEMENT] Upgrade support bundle kit version to v0.0.36 8162 - @c3y1huang @roger-ryao
• [BACKPORT][v1.6.1][IMPROVEMENT] Cannot read/write to block volume when the container is run as non-root 8122 - @PhanLe1010 @chriscchien
• [BACKPORT][v1.6.1][IMPROVEMENT] Improve environment_check script for NFS protocol bug and the host system self diagnosis 7971 - @james-munson @roger-ryao
• [BACKPORT][v1.6.1][IMPROVEMENT] Use HEAD instead of a GET to fetch the Content-Length of an resource via URL 7973 - @votdev @roger-ryao
• [BACKPORT][v1.6.1][IMPROVEMENT] Remove startup probe of CSI driver after liveness probe conn fix ready 7886 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][IMPROVEMENT] Change support-bundle-manager image pull policy to PullIfNotPresent 8000 - @ChanYiLin @roger-ryao

Bug

• [BACKPORT][v1.6.1][BUG] potential risk to unmap a negative number 8237 - @Vicente-Cheng @roger-ryao
• [BACKPORT][v1.6.1][BUG] Replica rebuild failed 8258 - @shuo-wu @roger-ryao
• [BACKPORT][v1.6.1][BUG] Can't use longhorn with Generic ephemeral volumes 8213 - @ejweber @chriscchien
• [BUG] [v1.6.1-rc1] v2 volume replica offline rebuilding fail 8187 - @shuo-wu @chriscchien
• [BACKPORT][v1.6.1][BUG] Longhorn api-server PUT request rate 8152 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][BUG] Use config map to update default-replica-count won't apply to default-replica-count.definition.default if the value equal to current default-replica-count.value 8134 - @james-munson @roger-ryao
• [BACKPORT][v1.6.1][BUG] Failed to restore a backup to file by the scripts/restore-backup-to-file.sh with a CIFS backup target. 8128 - @mantissahz @roger-ryao
• [BACKPORT][v1.6.1][BUG] Exporting data from the existing replicas has issues in 1.6.0 8096 - @ChanYiLin @chriscchien
• [BACKPORT][v1.6.1][BUG] longhorn manager pod fails to start in container-based K3s 7847 - @ChanYiLin @khushboo-rancher @chriscchien
• [BACKPORT][v1.6.1][BUG] no Pending workload pods for volume xxx to be mounted 8081 - @c3y1huang @chriscchien
• [BACKPORT][v1.6.1][BUG] Deadlock is possible in v1.6.0 instance manager 7920 - @roger-ryao
• [BACKPORT][v1.6.1][BUG][1.6.0] ENGINE v2 : disk /dev/xxxx is already used by AIO bdev disk-x 8133 - @derekbit @chriscchien
• [BACKPORT][v1.6.1][BUG] A replica may be incorrectly scheduled to a node with an existing failed replica 8044 - @ejweber @chriscchien
• [BACKPORT][v1.6.1][BUG] Deadlock between volume migration and upgrade after Longhorn upgrade 7870 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][BUG] Volume cannot attach because of the leftover non-empty volume.status.PendingNodeID after upgrading Longhorn 7995 - @james-munson @chriscchien
• [BACKPORT][v1.6.1][BUG] Longhorn may keep corrupted salvaged replicas and discard good ones 7885 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][BUG] LH manager reboots due to the webhook is not ready 8037 - @ChanYiLin @chriscchien
• [BACKPORT][v1.6.1][BUG] BackingImage does not download URL correctly in some situation 7987 - @votdev @yangchiu
• [BACKPORT][v1.6.1][BUG] Executing fstrim while rebuilding causes IO errors 7868 - @yangchiu @ejweber
• [BACKPORT][v1.6.1][BUG] The feature of auto remount read only volume not work on a single node cluster. 7846 - @yangchiu @ChanYiLin
• [BACKPORT][v1.6.1][BUG] Missed NodeStageVolume after reboot leads to CreateContainerError 8012 - @ejweber @chriscchien
• [BACKPORT][v1.6.1][BUG] persistence.removeSnapshotsDuringFilesystemTrim Helm variable is unreferenced 7952 - @ejweber @chriscchien
• [BACKPORT][v1.6.1][BUG][v1.5.4-rc4] Test case test_backuptarget_available_during_engine_image_not_ready failed to wait for backup target available 8055 - @c3y1huang @chriscchien
• [BACKPORT][v1.6.1][BUG] Add Snapshot Maximum Count to the Settings 7979 - @FrankYang0529 @roger-ryao
• [BACKPORT][v1.6.1][BUG] Volumes stuck upgrading after 1.5.3 -> 1.6.0 upgrade. 7899 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][BUG] The activated DR volume do not contain the latest data. 7946 - @shuo-wu @roger-ryao
• [BACKPORT][v1.6.1][BUG][v1.5.x] Recurring job fails to create backup when volume detached 8013 - @yangchiu @mantissahz @PhanLe1010 @c3y1huang
• [BACKPORT][v1.6.1][BUG] Create backup failed: failed lock lock-*.lck type 1 acquisition 7875 - @yangchiu @ChanYiLin @chriscchien
• [BUG] Fix errors in questions.yaml 6392 - @james-munson @chriscchien

Misc

• [BACKPORT][v1.6.1][REFACTOR] move mount point check function to common lib 8124 - @ChanYiLin @chriscchien
• [TASK] Fix updating patch digest dependencies (v1.6.x) 8107 - @mantissahz

Contributors

• @ChanYiLin
• @FrankYang0529
• @PhanLe1010
• @Vicente-Cheng
• @c3y1huang
• @chriscchien
• @derekbit
• @ejweber
• @innobead
• @james-munson
• @khushboo-rancher
• @mantissahz
• @roger-ryao
• @shuo-wu
• @votdev
• @yangchiu

Longhorn v1.6.1-rc2

Warning

DON'T UPGRADE from/to any RC/Preview/Sprint releases because the operation is not supported.

Resolved Issues in this release

Improvement

• [BACKPORT][v1.6.1][IMPROVEMENT] Upgrade support bundle kit version to v0.0.36 8162 - @c3y1huang @roger-ryao
• [BACKPORT][v1.6.1][IMPROVEMENT] Allow users to request backup volume update 7984 - @mantissahz
• [BACKPORT][v1.6.1][IMPROVEMENT] Cannot read/write to block volume when the container is run as non-root 8122 - @PhanLe1010 @chriscchien
• [BACKPORT][v1.6.1][IMPROVEMENT] Improve environment_check script for NFS protocol bug and the host system self diagnosis 7971 - @james-munson @roger-ryao
• [BACKPORT][v1.6.1][IMPROVEMENT] Use HEAD instead of a GET to fetch the Content-Length of an resource via URL 7973 - @votdev @roger-ryao
• [BACKPORT][v1.6.1][IMPROVEMENT] Remove startup probe of CSI driver after liveness probe conn fix ready 7886 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][IMPROVEMENT] Change support-bundle-manager image pull policy to PullIfNotPresent 8000 - @ChanYiLin @roger-ryao

Bug

• [BUG] [v1.6.1-rc1] v2 volume replica offline rebuilding fail 8187 - @shuo-wu @chriscchien
• [BACKPORT][v1.6.1][BUG] Longhorn api-server PUT request rate 8152 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][BUG] Use config map to update default-replica-count won't apply to default-replica-count.definition.default if the value equal to current default-replica-count.value 8134 - @james-munson @roger-ryao
• [BACKPORT][v1.6.1][BUG] Failed to restore a backup to file by the scripts/restore-backup-to-file.sh with a CIFS backup target. 8128 - @mantissahz @roger-ryao
• [BACKPORT][v1.6.1][BUG] Exporting data from the existing replicas has issues in 1.6.0 8096 - @ChanYiLin @chriscchien
• [BACKPORT][v1.6.1][BUG] longhorn manager pod fails to start in container-based K3s 7847 - @ChanYiLin @khushboo-rancher @chriscchien
• [BACKPORT][v1.6.1][BUG] no Pending workload pods for volume xxx to be mounted 8081 - @c3y1huang @chriscchien
• [BACKPORT][v1.6.1][BUG] Deadlock is possible in v1.6.0 instance manager 7920 - @roger-ryao
• [BACKPORT][v1.6.1][BUG] metric longhorn_volume_robustness did not reflect detached volume 8140 - @c3y1huang
• [BACKPORT][v1.6.1][BUG][1.6.0] ENGINE v2 : disk /dev/xxxx is already used by AIO bdev disk-x 8133 - @derekbit @chriscchien
• [BACKPORT][v1.6.1][BUG] A replica may be incorrectly scheduled to a node with an existing failed replica 8044 - @ejweber @chriscchien
• [BACKPORT][v1.6.1][BUG] Deadlock between volume migration and upgrade after Longhorn upgrade 7870 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][BUG] Volume cannot attach because of the leftover non-empty volume.status.PendingNodeID after upgrading Longhorn 7995 - @james-munson @chriscchien
• [BACKPORT][v1.6.1][BUG] Longhorn may keep corrupted salvaged replicas and discard good ones 7885 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][BUG] LH manager reboots due to the webhook is not ready 8037 - @ChanYiLin @chriscchien
• [BACKPORT][v1.6.1][BUG] BackingImage does not download URL correctly in some situation 7987 - @votdev @yangchiu
• [BACKPORT][v1.6.1][BUG] Executing fstrim while rebuilding causes IO errors 7868 - @yangchiu @ejweber
• [BACKPORT][v1.6.1][BUG] The feature of auto remount read only volume not work on a single node cluster. 7846 - @yangchiu @ChanYiLin
• [BACKPORT][v1.6.1][BUG] Missed NodeStageVolume after reboot leads to CreateContainerError 8012 - @ejweber @chriscchien
• [BACKPORT][v1.6.1][BUG] persistence.removeSnapshotsDuringFilesystemTrim Helm variable is unreferenced 7952 - @ejweber @chriscchien
• [BACKPORT][v1.6.1][BUG][v1.5.4-rc4] Test case test_backuptarget_available_during_engine_image_not_ready failed to wait for backup target available 8055 - @c3y1huang @chriscchien
• [BACKPORT][v1.6.1][BUG] Add Snapshot Maximum Count to the Settings 7979 - @FrankYang0529 @roger-ryao
• [BACKPORT][v1.6.1][BUG] Volumes stuck upgrading after 1.5.3 -> 1.6.0 upgrade. 7899 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][BUG] The activated DR volume do not contain the latest data. 7946 - @shuo-wu @roger-ryao
• [BACKPORT][v1.6.1][BUG][v1.5.x] Recurring job fails to create backup when volume detached 8013 - @yangchiu @mantissahz @PhanLe1010 @c3y1huang
• [BACKPORT][v1.6.1][BUG] Create backup failed: failed lock lock-*.lck type 1 acquisition 7875 - @yangchiu @ChanYiLin @chriscchien
• [BACKPORT][v1.6.1][BUG][v1.5.4-rc1] Recurring job failed to create/delete backups after node reboot 7855 - @james-munson
• [BUG] Fix errors in questions.yaml 6392 - @james-munson @chriscchien

Misc

• [BACKPORT][v1.6.1][REFACTOR] move mount point check function to common lib 8124 - @ChanYiLin @chriscchien
• [TASK] Fix updating patch digest dependencies (v1.6.x) 8107 - @mantissahz

Contributors

• @ChanYiLin
• @FrankYang0529
• @PhanLe1010
• @c3y1huang
• @chriscchien
• @derekbit
• @ejweber
• @innobead
• @james-munson
• @khushboo-rancher
• @mantissahz
• @roger-ryao
• @shuo-wu
• @votdev
• @yangchiu

Longhorn v1.6.1-rc1

DON'T UPGRADE from/to any RC/Preview/Sprint releases because the operation is not supported.

Resolved Issues in this release

Improvement

• [BACKPORT][v1.6.1][IMPROVEMENT] Upgrade support bundle kit version to v0.0.36 8162 - @c3y1huang @roger-ryao
• [BACKPORT][v1.6.1][IMPROVEMENT] Allow users to request backup volume update 7984 - @mantissahz
• [BACKPORT][v1.6.1][IMPROVEMENT] Cannot read/write to block volume when the container is run as non-root 8122 - @PhanLe1010 @chriscchien
• [BACKPORT][v1.6.1][IMPROVEMENT] Improve environment_check script for NFS protocol bug and the host system self diagnosis 7971 - @james-munson @roger-ryao
• [BACKPORT][v1.6.1][IMPROVEMENT] Use HEAD instead of a GET to fetch the Content-Length of an resource via URL 7973 - @votdev @roger-ryao
• [BACKPORT][v1.6.1][IMPROVEMENT] Remove startup probe of CSI driver after liveness probe conn fix ready 7886 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][IMPROVEMENT] Change support-bundle-manager image pull policy to PullIfNotPresent 8000 - @ChanYiLin @roger-ryao

Bug

• [BACKPORT][v1.6.1][BUG] Exporting data from the existing replicas has issues in 1.6.0 8096 - @ChanYiLin @chriscchien
• [BACKPORT][v1.6.1][BUG] Longhorn api-server PUT request rate 8152 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][BUG] no Pending workload pods for volume xxx to be mounted 8081 - @c3y1huang @chriscchien
• [BACKPORT][v1.6.1][BUG] Deadlock is possible in v1.6.0 instance manager 7920 - @roger-ryao
• [BACKPORT][v1.6.1][BUG] metric longhorn_volume_robustness did not reflect detached volume 8140 - @c3y1huang
• [BACKPORT][v1.6.1][BUG][1.6.0] ENGINE v2 : disk /dev/xxxx is already used by AIO bdev disk-x 8133 - @derekbit @chriscchien
• [BACKPORT][v1.6.1][BUG] A replica may be incorrectly scheduled to a node with an existing failed replica 8044 - @ejweber @chriscchien
• [BACKPORT][v1.6.1][BUG] Deadlock between volume migration and upgrade after Longhorn upgrade 7870 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][BUG] Volume cannot attach because of the leftover non-empty volume.status.PendingNodeID after upgrading Longhorn 7995 - @james-munson @chriscchien
• [BACKPORT][v1.6.1][BUG] Use config map to update default-replica-count won't apply to default-replica-count.definition.default if the value equal to current default-replica-count.value 8134 - @james-munson @roger-ryao
• [BACKPORT][v1.6.1][BUG] Failed to restore a backup to file by the scripts/restore-backup-to-file.sh with a CIFS backup target. 8128 - @mantissahz @roger-ryao
• [BACKPORT][v1.6.1][BUG] longhorn manager pod fails to start in container-based K3s 7847 - @ChanYiLin @khushboo-rancher @chriscchien
• [BACKPORT][v1.6.1][BUG] Longhorn may keep corrupted salvaged replicas and discard good ones 7885 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][BUG] LH manager reboots due to the webhook is not ready 8037 - @ChanYiLin @chriscchien
• [BACKPORT][v1.6.1][BUG] BackingImage does not download URL correctly in some situation 7987 - @votdev @yangchiu
• [BACKPORT][v1.6.1][BUG] Executing fstrim while rebuilding causes IO errors 7868 - @yangchiu @ejweber
• [BACKPORT][v1.6.1][BUG] The feature of auto remount read only volume not work on a single node cluster. 7846 - @yangchiu @ChanYiLin
• [BACKPORT][v1.6.1][BUG] Missed NodeStageVolume after reboot leads to CreateContainerError 8012 - @ejweber @chriscchien
• [BACKPORT][v1.6.1][BUG] persistence.removeSnapshotsDuringFilesystemTrim Helm variable is unreferenced 7952 - @ejweber @chriscchien
• [BACKPORT][v1.6.1][BUG][v1.5.4-rc4] Test case test_backuptarget_available_during_engine_image_not_ready failed to wait for backup target available 8055 - @c3y1huang @chriscchien
• [BACKPORT][v1.6.1][BUG] Add Snapshot Maximum Count to the Settings 7979 - @FrankYang0529 @roger-ryao
• [BACKPORT][v1.6.1][BUG] Volumes stuck upgrading after 1.5.3 -> 1.6.0 upgrade. 7899 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][BUG] The activated DR volume do not contain the latest data. 7946 - @shuo-wu @roger-ryao
• [BACKPORT][v1.6.1][BUG][v1.5.x] Recurring job fails to create backup when volume detached 8013 - @yangchiu @mantissahz @PhanLe1010 @c3y1huang
• [BACKPORT][v1.6.1][BUG] Create backup failed: failed lock lock-*.lck type 1 acquisition 7875 - @yangchiu @ChanYiLin @chriscchien
• [BACKPORT][v1.6.1][BUG][v1.5.4-rc1] Recurring job failed to create/delete backups after node reboot 7855 - @james-munson
• [BUG] Fix errors in questions.yaml 6392 - @james-munson @chriscchien

Misc

• [BACKPORT][v1.6.1][REFACTOR] move mount point check function to common lib 8124 - @ChanYiLin @chriscchien
• [TASK] Fix updating patch digest dependencies (v1.6.x) 8107 - @mantissahz

Contributors

• @ChanYiLin
• @FrankYang0529
• @PhanLe1010
• @c3y1huang
• @chriscchien
• @derekbit
• @ejweber
• @innobead
• @james-munson
• @khushboo-rancher
• @mantissahz
• @roger-ryao
• @shuo-wu
• @votdev
• @yangchiu

Longhorn v1.5.4

Longhorn v1.5.4 Release Notes

This latest stable version of Longhorn 1.5 introduces several improvements and bug fixes that are intended to improve system quality, resilience, and stability.

The Longhorn team appreciates your contributions and anticipates receiving feedback regarding this release.

Note:
For more information about release-related terminology, see Releases.

Installation

Ensure that your cluster is running Kubernetes v1.21 or later before installing Longhorn v1.5.4.

You can install Longhorn using a variety of tools, including Rancher, Kubectl, and Helm. For more information about installation methods and requirements, see Quick Installation in the Longhorn documentation.

Upgrade

Ensure that your cluster is running Kubernetes v1.21 or later before upgrading from Longhorn v1.4.x to v1.5.4.

Longhorn only allows upgrades from supported versions. For more information about upgrade paths and procedures, see Upgrade in the Longhorn documentation.

Deprecation & Incompatibilities

For information about important changes, including feature incompatibility, deprecation, and removal, see Important Notes in the Longhorn documentation.

Post-Release Known Issues

For information about issues identified after this release, see Release-Known-Issues.

Resolved Issues

Highlight

• [BACKPORT][v1.5.4][FEATURE] Add a new settings that allows Longhorn to evict replicas automatically when a node is drained 7421 - @yangchiu @ejweber

Improvement

• [BACKPORT][v1.5.4][IMPROVEMENT] Use HEAD instead of a GET to fetch the Content-Length of an resource via URL 7981 - @votdev @roger-ryao
• [BACKPORT][v1.5.4][IMPROVEMENT] Change support-bundle-manager image pull policy to PullIfNotPresent 7999 - @ChanYiLin @chriscchien
• [BACKPORT][v1.5.4][FEATURE] Update base image of Longhorn components to BCI 15.5 7134 - @nitendra-suse
• [BACKPORT][v1.5.4]Allow to set mount options for storageclass via values.yaml in helm chart 7593 - @ChanYiLin @mantissahz
• [BACKPORT][v1.5.4][IMPROVEMENT] Remove startup probe of CSI driver after liveness probe conn fix ready 7933 - @ejweber @chriscchien
• [IMPROVEMENT] Make environment_check look for a global default K8s priority class in those releases that it affects. 7831 - @mantissahz @james-munson
• [BACKPORT][v1.5.4][IMPROVEMENT] Allow deployment of Prometheus ServiceMonitor with the Longhorn helm chart 7864 - @mantissahz @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] Remove unused process manager connection in longhorn-manager 7785 - @derekbit @roger-ryao
• [BACKPORT][v1.5.4][IMPROVEMENT] Clean up backup target in IM-R pod if the backup target setting is unset 7145 - @ChanYiLin @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] BackingImage should be compressed when downloading and use the name as filename instead of UUID 7397 - @ChanYiLin @roger-ryao
• [BACKPORT][v1.5.4][IMPROVEMENT] Automatically remount read-only RWO volume to read-write 7500 - @ChanYiLin @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] deploy: driver deployer shouldn't cleanup previous deployment if Kubernetes version changes 7345 - @PhanLe1010 @roger-ryao
• [BACKPORT][v1.5.4][IMPROVEMENT] Only restarts pods with volumes in the unexpected Read-Only state 7729 - @yangchiu @ChanYiLin
• [BACKPORT][v1.5.4][IMPROVEMENT] Improve handling of 16TiB+ volumes with ext4 as the underlying file system 7429 - @mantissahz @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] Volumes: metrics for snapshots include (size and type: system vs user) 7725 - @c3y1huang @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] Improve the profiler of longhorn-engine for runtime profiling 7545 - @Vicente-Cheng @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] Don't crash the migration engine when kubelet restarts 7328 - @yangchiu @ejweber
• [BACKPORT][v1.5.4][IMPROVEMENT] Upgrade CSI components to the latest patch release 7492 - @c3y1huang @roger-ryao
• [BACKPORT][v1.5.4][IMPROVEMENT] Reject the last replica deletion if its volume.spec.deletionTimestamp is not set 7432 - @yangchiu @derekbit
• [BACKPORT][v1.5.4][IMPROVEMENT] Upgrade support bundle kit version to v0.0.33 7279 - @c3y1huang
• [BACKPORT][v1.5.4][IMPROVEMENT] Review and simplify longhorn component image build 7162 - @ChanYiLin
• [BACKPORT][v1.5.4][IMPROVEMENT] Replace deprecated grpc.WithInsecure 7364 - @c3y1huang
• [BACKPORT][v1.5.4][IMPROVEMENT] Have a setting to disable snapshot purge for maintenance purpose 7265 - @ejweber @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] Bypass upgrade when installing a fresh setup 7283 - @mantissahz @roger-ryao

Bug

• [BACKPORT][v1.5.4][BUG][v1.5.4-rc4] Test case test_backuptarget_available_during_engine_image_not_ready failed to wait for backup target available 8054 - @yangchiu @c3y1huang
• [BACKPORT][v1.5.4][BUG] The activated DR volume do not contain the latest data. 7947 - @shuo-wu @roger-ryao
• [BUG][v1.5.x] DR volume unable to be activated if the latest backup's been deleted 7997 - @yangchiu @shuo-wu
• [BUG] Backup related test cases failed 7989 - @yangchiu @shuo-wu
• [BACKPORT][v1.5.4][BUG][v1.5.x] Recurring job fails to create backup when volume detached 8015 - @yangchiu @mantissahz @PhanLe1010 @c3y1huang
• [BACKPORT][v1.5.4][BUG] Deadlock for RWX volume if an error occurs in its share-manager pod 7186 - @ejweber @chriscchien
• [BACKPORT][v1.5.4][BUG] Deadlock is possible in v1.6.0 instance manager 7941 - @ejweber @roger-ryao
• [BACKPORT][v1.5.4][BUG] Longhorn may keep corrupted salvaged replicas and discard good ones 7801 - @ejweber @chriscchien
• [BACKPORT][v1.5.4][BUG] Deadlock between volume migration and upgrade after Longhorn upgrade 7869 - @ejweber @chriscchien
• [BACKPORT][v1.5.4][BUG] Executing fstrim while rebuilding causes IO errors 7867 - @ejweber @chriscchien
• [BACKPORT][v1.5.4][BUG] BackingImage does not download URL correctly in some situation 7986 - @yangchiu
• [BACKPORT][v1.5.4][BUG] The feature of auto remount read only volume not work on a single node cluster. 7844 - @ChanYiLin @chriscchien
• [BACKPORT][v1.5.4][BUG] Volumes stuck upgrading after 1.5.3 -> 1.6.0 upgrade. 7901 - @yangchiu @ejweber
• [BUG][v1.5.4-rc1] V2 volume have engine upgrade option on UI after upgrade from v1.5.3 to v1.5.4-rc1 7863 - @chriscchien @scures
• [BACKPORT][v1.5.4][BUG] longhorn manager pod fails to start in container-based K3s 7848 - @ChanYiLin
• [BACKPORT][v1.5.4][BUG] Relax S3 client retry intervals, for throttled requests 7098 - @mantissahz @chriscchien
• [BACKPORT][v1.5.4][BUG][v1.6.0-rc1] Negative test case failed: Stop Volume Node Kubelet For More Than Pod Eviction Timeout While Workload Heavy Writing 7761 - @yangchiu @c3y1huang
• [BACKPORT][v1.5.4][BUG] Volumes don't mount with mTLS enabled 7789 - @derekbit @roger-ryao
• [BACKPORT][v1.5.4][BUG] supportbundle/kubelet.log empty in k3s environment 7123 - @c3y1huang @chriscchien @roger-ryao
• [BUG] v1.5.x/v1.4.x BackingImage download fails if URL has query parameters 7822 - @ChanYiLin @mantissahz
• [BACKPORT][v1.5.4][BUG] Metric totalVolumeSize and totalVolumeActualSize incorrect due to v2 volume counts 7392 - @c3y1huang @chriscchien
• [BACKPORT][v1.5.4][BUG] Continuously auto-balancing replicas when zone does not have enough space [7306](https://github.com/longh...

Longhorn v1.5.4-rc4

DON'T UPGRADE from/to any RC/Preview/Sprint releases because the operation is not supported.

Resolved Issues in this release

Highlight

• [BACKPORT][v1.5.4][FEATURE] Add a new settings that allows Longhorn to evict replicas automatically when a node is drained 7421 - @yangchiu @ejweber

Improvement

• [BACKPORT][v1.5.4][IMPROVEMENT] Use HEAD instead of a GET to fetch the Content-Length of an resource via URL 7981 - @votdev @roger-ryao
• [BACKPORT][v1.5.4][IMPROVEMENT] Change support-bundle-manager image pull policy to PullIfNotPresent 7999 - @ChanYiLin @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] Use HEAD instead of a GET to fetch the Content-Length of an resource via URL 7974 - @votdev
• [BACKPORT][v1.5.4][FEATURE] Update base image of Longhorn components to BCI 15.5 7134 - @nitendra-suse
• [BACKPORT][v1.5.4]Allow to set mount options for storageclass via values.yaml in helm chart 7593 - @ChanYiLin @mantissahz
• [BACKPORT][v1.5.4][IMPROVEMENT] Remove startup probe of CSI driver after liveness probe conn fix ready 7933 - @ejweber @chriscchien
• [IMPROVEMENT] Make environment_check look for a global default K8s priority class in those releases that it affects. 7831 - @mantissahz @james-munson
• [BACKPORT][v1.5.4][IMPROVEMENT] Allow deployment of Prometheus ServiceMonitor with the Longhorn helm chart 7864 - @mantissahz @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] Remove unused process manager connection in longhorn-manager 7785 - @derekbit @roger-ryao
• [BACKPORT][v1.5.4][IMPROVEMENT] Clean up backup target in IM-R pod if the backup target setting is unset 7145 - @ChanYiLin @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] BackingImage should be compressed when downloading and use the name as filename instead of UUID 7397 - @ChanYiLin @roger-ryao
• [BACKPORT][v1.5.4][IMPROVEMENT] Automatically remount read-only RWO volume to read-write 7500 - @ChanYiLin @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] deploy: driver deployer shouldn't cleanup previous deployment if Kubernetes version changes 7345 - @PhanLe1010 @roger-ryao
• [BACKPORT][v1.5.4][IMPROVEMENT] Only restarts pods with volumes in the unexpected Read-Only state 7729 - @yangchiu @ChanYiLin
• [BACKPORT][v1.5.4][IMPROVEMENT] Improve handling of 16TiB+ volumes with ext4 as the underlying file system 7429 - @mantissahz @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] Volumes: metrics for snapshots include (size and type: system vs user) 7725 - @c3y1huang @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] Improve the profiler of longhorn-engine for runtime profiling 7545 - @Vicente-Cheng @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] Don't crash the migration engine when kubelet restarts 7328 - @yangchiu @ejweber
• [BACKPORT][v1.5.4][IMPROVEMENT] Upgrade CSI components to the latest patch release 7492 - @c3y1huang @roger-ryao
• [BACKPORT][v1.5.4][IMPROVEMENT] Reject the last replica deletion if its volume.spec.deletionTimestamp is not set 7432 - @yangchiu @derekbit
• [BACKPORT][v1.5.4][IMPROVEMENT] Upgrade support bundle kit version to v0.0.33 7279 - @c3y1huang
• [BACKPORT][v1.5.4][IMPROVEMENT] Review and simplify longhorn component image build 7162 - @ChanYiLin
• [BACKPORT][v1.5.4][IMPROVEMENT] Replace deprecated grpc.WithInsecure 7364 - @c3y1huang
• [BACKPORT][v1.5.4][IMPROVEMENT] Have a setting to disable snapshot purge for maintenance purpose 7265 - @ejweber @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] Bypass upgrade when installing a fresh setup 7283 - @mantissahz @roger-ryao

Bug

• [BUG][v1.5.x] DR volume unable to be activated if the latest backup's been deleted 7997 - @yangchiu @shuo-wu
• [BACKPORT][v1.5.4][BUG][v1.5.x] Recurring job fails to create backup when volume detached 8015 - @yangchiu @mantissahz @PhanLe1010 @c3y1huang
• [BACKPORT][v1.5.4][BUG] The activated DR volume do not contain the latest data. 7947 - @shuo-wu @roger-ryao
• [BACKPORT][v1.5.4][BUG] Deadlock for RWX volume if an error occurs in its share-manager pod 7186 - @ejweber @chriscchien
• [BACKPORT][v1.5.4][BUG] Deadlock is possible in v1.6.0 instance manager 7941 - @ejweber @roger-ryao
• [BACKPORT][v1.5.4][BUG] Longhorn may keep corrupted salvaged replicas and discard good ones 7801 - @ejweber @chriscchien
• [BACKPORT][v1.5.4][BUG] Deadlock between volume migration and upgrade after Longhorn upgrade 7869 - @ejweber @chriscchien
• [BACKPORT][v1.5.4][BUG] Executing fstrim while rebuilding causes IO errors 7867 - @ejweber @chriscchien
• [BUG] Backup related test cases failed 7989 - @yangchiu @shuo-wu
• [BACKPORT][v1.5.4][BUG] BackingImage does not download URL correctly in some situation 7986 - @yangchiu
• [BACKPORT][v1.5.4][BUG] The feature of auto remount read only volume not work on a single node cluster. 7844 - @ChanYiLin @chriscchien
• [BACKPORT][v1.5.4][BUG] Volumes stuck upgrading after 1.5.3 -> 1.6.0 upgrade. 7901 - @yangchiu @ejweber
• [BUG][v1.5.4-rc1] V2 volume have engine upgrade option on UI after upgrade from v1.5.3 to v1.5.4-rc1 7863 - @chriscchien @scures
• [BACKPORT][v1.5.4][BUG] longhorn manager pod fails to start in container-based K3s 7848 - @ChanYiLin
• [BACKPORT][v1.5.4][BUG] Relax S3 client retry intervals, for throttled requests 7098 - @mantissahz @chriscchien
• [BACKPORT][v1.5.4][BUG][v1.6.0-rc1] Negative test case failed: Stop Volume Node Kubelet For More Than Pod Eviction Timeout While Workload Heavy Writing 7761 - @yangchiu @c3y1huang
• [BACKPORT][v1.5.4][BUG] Volumes don't mount with mTLS enabled 7789 - @derekbit @roger-ryao
• [BACKPORT][v1.5.4][BUG] supportbundle/kubelet.log empty in k3s environment 7123 - @c3y1huang @chriscchien @roger-ryao
• [BUG] v1.5.x/v1.4.x BackingImage download fails if URL has query parameters 7822 - @ChanYiLin @mantissahz
• [BACKPORT][v1.5.4][BUG] Metric totalVolumeSize and totalVolumeActualSize incorrect due to v2 volume counts 7392 - @c3y1huang @chriscchien
• [BACKPORT][v1.5.4][BUG] Continuously auto-balancing replicas when zone does not have enough space 7306 - @c3y1huang @chriscchien
• [BACKPORT][v1.5.4][BUG] Unable to list backups when backuptarget resource is picked up by a cordoned node 7621 - @mantissahz @c3y1huang
• [BUG] The wrong template in default-setting.yaml of the Longhorn chart in v1.5 and v1.4 7459 - @mantissahz @roger-ryao
• [BACKPORT][v1.5.4][BUG] Failed to check_volume_data after volume engine upgrade/migration 7402 - @PhanLe1010 @chriscchien
• [BACKPORT][v1.5.4][BUG] Volume conditions are not represented in the UI for v1.4.x and newer 7242 - @m-ildefons @roger-ryao
• [BACKPORT][v1.5.4][BUG] Confusing logging when trying to attach a new volume with no scheduled replicas 7245 - @ejweber @roger-ryao
• [BACKPORT][v1.5.4][BUG] Environment check script claims success when kubectl fails. 7216 - @james-munson @roger-ryao
• [BACKPORT][v1.5.4][BUG] Backup volume attachment tickets might not be cleaned up after completion. 7604 - @james-munson @chriscchien
• [BACKPORT][v1.5.4][BUG][v1.6.0-rc1] Some Longhorn resources remaining after longhorn-uninstall job completed 7663 - @yangchiu @PhanLe1010
• [BACKPORT][v1.5.4][BUG] Backi...

Longhorn v1.5.4-rc3

DON'T UPGRADE from/to any RC/Preview/Sprint releases because the operation is not supported.

Resolved Issues in this release

Highlight

• [BACKPORT][v1.5.4][FEATURE] Add a new settings that allows Longhorn to evict replicas automatically when a node is drained 7421 - @yangchiu @ejweber

Improvement

• [BACKPORT][v1.5.4]Allow to set mount options for storageclass via values.yaml in helm chart 7593 - @ChanYiLin @mantissahz
• [BACKPORT][v1.5.4][FEATURE] Update base image of Longhorn components to BCI 15.5 7134 - @nitendra-suse
• [BACKPORT][v1.5.4][IMPROVEMENT] Use HEAD instead of a GET to fetch the Content-Length of an resource via URL 7981 - @votdev @roger-ryao
• [BACKPORT][v1.5.4][IMPROVEMENT] Use HEAD instead of a GET to fetch the Content-Length of an resource via URL 7974 - @votdev
• [BACKPORT][v1.5.4][IMPROVEMENT] Improve environment_check script for NFS protocol bug and the host system self diagnosis 7969 - @james-munson
• [BACKPORT][v1.5.4][IMPROVEMENT] Remove startup probe of CSI driver after liveness probe conn fix ready 7933 - @ejweber @chriscchien
• [IMPROVEMENT] Make environment_check look for a global default K8s priority class in those releases that it affects. 7831 - @mantissahz @james-munson
• [BACKPORT][v1.5.4][IMPROVEMENT] Allow deployment of Prometheus ServiceMonitor with the Longhorn helm chart 7864 - @mantissahz @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] Remove unused process manager connection in longhorn-manager 7785 - @derekbit @roger-ryao
• [BACKPORT][v1.5.4][IMPROVEMENT] Clean up backup target in IM-R pod if the backup target setting is unset 7145 - @ChanYiLin @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] BackingImage should be compressed when downloading and use the name as filename instead of UUID 7397 - @ChanYiLin @roger-ryao
• [BACKPORT][v1.5.4][IMPROVEMENT] Automatically remount read-only RWO volume to read-write 7500 - @ChanYiLin @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] deploy: driver deployer shouldn't cleanup previous deployment if Kubernetes version changes 7345 - @PhanLe1010 @roger-ryao
• [BACKPORT][v1.5.4][IMPROVEMENT] Only restarts pods with volumes in the unexpected Read-Only state 7729 - @yangchiu @ChanYiLin
• [BACKPORT][v1.5.4][IMPROVEMENT] Improve handling of 16TiB+ volumes with ext4 as the underlying file system 7429 - @mantissahz @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] Volumes: metrics for snapshots include (size and type: system vs user) 7725 - @c3y1huang @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] Improve the profiler of longhorn-engine for runtime profiling 7545 - @Vicente-Cheng @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] Don't crash the migration engine when kubelet restarts 7328 - @yangchiu @ejweber
• [BACKPORT][v1.5.4][IMPROVEMENT] Upgrade CSI components to the latest patch release 7492 - @c3y1huang @roger-ryao
• [BACKPORT][v1.5.4][IMPROVEMENT] Reject the last replica deletion if its volume.spec.deletionTimestamp is not set 7432 - @yangchiu @derekbit
• [BACKPORT][v1.5.4][IMPROVEMENT] Upgrade support bundle kit version to v0.0.33 7279 - @c3y1huang
• [BACKPORT][v1.5.4][IMPROVEMENT] Review and simplify longhorn component image build 7162 - @ChanYiLin
• [BACKPORT][v1.5.4][IMPROVEMENT] Replace deprecated grpc.WithInsecure 7364 - @c3y1huang
• [BACKPORT][v1.5.4][IMPROVEMENT] Have a setting to disable snapshot purge for maintenance purpose 7265 - @ejweber @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] Bypass upgrade when installing a fresh setup 7283 - @mantissahz @roger-ryao

Bug

• [BUG][v1.5.x] Recurring job fails to create backup when volume detached 7937 - @mantissahz @c3y1huang
• [BUG] Backup related test cases failed 7989 - @yangchiu @shuo-wu
• [BACKPORT][v1.5.4][BUG] The activated DR volume do not contain the latest data. 7947 - @shuo-wu @roger-ryao
• [BACKPORT][v1.5.4][BUG] BackingImage does not download URL correctly in some situation 7986 - @yangchiu
• [BACKPORT][v1.5.4][BUG] The feature of auto remount read only volume not work on a single node cluster. 7844 - @ChanYiLin @chriscchien
• [BACKPORT][v1.5.4][BUG] Deadlock is possible in v1.6.0 instance manager 7941 - @ejweber @roger-ryao
• [BACKPORT][v1.5.4][BUG] Volumes stuck upgrading after 1.5.3 -> 1.6.0 upgrade. 7901 - @yangchiu @ejweber
• [BUG][v1.5.4-rc1] V2 volume have engine upgrade option on UI after upgrade from v1.5.3 to v1.5.4-rc1 7863 - @chriscchien @scures
• [BACKPORT][v1.5.4][BUG] longhorn manager pod fails to start in container-based K3s 7848 - @ChanYiLin
• [BACKPORT][v1.5.4][BUG] Deadlock between volume migration and upgrade after Longhorn upgrade 7869 - @ejweber @chriscchien
• [BACKPORT][v1.5.4][BUG] Executing fstrim while rebuilding causes IO errors 7867 - @ejweber @chriscchien
• [BACKPORT][v1.5.4][BUG] Relax S3 client retry intervals, for throttled requests 7098 - @mantissahz @chriscchien
• [BACKPORT][v1.5.4][BUG][v1.6.0-rc1] Negative test case failed: Stop Volume Node Kubelet For More Than Pod Eviction Timeout While Workload Heavy Writing 7761 - @yangchiu @c3y1huang
• [BACKPORT][v1.5.4][BUG] Volumes don't mount with mTLS enabled 7789 - @derekbit @roger-ryao
• [BACKPORT][v1.5.4][BUG] supportbundle/kubelet.log empty in k3s environment 7123 - @c3y1huang @chriscchien @roger-ryao
• [BACKPORT][v1.5.4][BUG] Longhorn may keep corrupted salvaged replicas and discard good ones 7801 - @ejweber @chriscchien
• [BUG] v1.5.x/v1.4.x BackingImage download fails if URL has query parameters 7822 - @ChanYiLin @mantissahz
• [BACKPORT][v1.5.4][BUG] Metric totalVolumeSize and totalVolumeActualSize incorrect due to v2 volume counts 7392 - @c3y1huang @chriscchien
• [BACKPORT][v1.5.4][BUG] Continuously auto-balancing replicas when zone does not have enough space 7306 - @c3y1huang @chriscchien
• [BACKPORT][v1.5.4][BUG] Unable to list backups when backuptarget resource is picked up by a cordoned node 7621 - @mantissahz @c3y1huang
• [BUG] The wrong template in default-setting.yaml of the Longhorn chart in v1.5 and v1.4 7459 - @mantissahz @roger-ryao
• [BACKPORT][v1.5.4][BUG] Failed to check_volume_data after volume engine upgrade/migration 7402 - @PhanLe1010 @chriscchien
• [BACKPORT][v1.5.4][BUG] Volume conditions are not represented in the UI for v1.4.x and newer 7242 - @m-ildefons @roger-ryao
• [BACKPORT][v1.5.4][BUG] Confusing logging when trying to attach a new volume with no scheduled replicas 7245 - @ejweber @roger-ryao
• [BACKPORT][v1.5.4][BUG] Environment check script claims success when kubectl fails. 7216 - @james-munson @roger-ryao
• [BACKPORT][v1.5.4][BUG] Backup volume attachment tickets might not be cleaned up after completion. 7604 - @james-munson @chriscchien
• [BACKPORT][v1.5.4][BUG][v1.6.0-rc1] Some Longhorn resources remaining after longhorn-uninstall job completed 7663 - @yangchiu @PhanLe1010
• [BACKPORT][v1.5.4][BUG] Backing Image Data Inconsistency if it's Exported from a Backing Image Backed Volume 7701 - @yangchiu @ChanYiLin
• [BACKPORT][v1.5.4][BUG] CSI components CrashLoopBackOff, failed to connect to unix://csi/csi.sock after cluster restart 7426 - @ejweber @roger-ryao
• [BA...