Longhorn v1.6.0-rc1

DON'T UPGRADE to/from any RC/Preview/Sprint releases because the operation is not supported!

Changes in this release

Highlight

• [FEATURE] Support eventual danger zone setting update 7173 - @mantissahz
• [FEATURE] Engine upgrade enforcement 5842 - @yangchiu @c3y1huang
• [FEATURE] Support instance-manager pod for v2 volumes on selected nodes 7015 - @derekbit @chriscchien @roger-ryao
• [FEATURE] Longhorn snapshot space management 6563 - @FrankYang0529 @yangchiu
• [FEATURE] v2 volume supports volume backup/restore 6138 - @yangchiu @derekbit
• [IMPROVEMENT] Remove or Change Helm pre-upgrade hook to support ArgoCD 6415 - @mantissahz
• [FEATURE] Restore BackingImage for BackupVolume in a new cluster 4165 - @ChanYiLin @roger-ryao
• [FEATURE] Talos support 3161 - @yangchiu @c3y1huang
• [FEATURE] Support v2 volume on ARM64 platform 6021 - @derekbit @chriscchien @roger-ryao
• [FEATURE] Have default priorityClass to prevent unexpected longhorn pods eviction 6528 - @mantissahz
• [IMPROVEMENT] Add a new settings that allows Longhorn to evict replicas automatically when a node is drained 2238 - @ejweber @chriscchien
• [FEATURE] Add linear dm device on the top of v2 volume 7357 - @derekbit @chriscchien
• [TASK] Bump up the minimum supported Kubernetes version 7224 - @c3y1huang @roger-ryao
• [FEATURE] Support Encryption for VolumeMode Block 4883 - @derekbit @roger-ryao
• [TASK] Bump the versions of dependent libs or components 7001 - @c3y1huang @chriscchien
• [TASK] Add install/upgrade longhorn by gitops (flux) pipeline 6343 - @yangchiu
• [FEATURE] OKD/Openshift support 1831 - @mantissahz @ArthurVardevanyan @roger-ryao

Feature

• [FEATURE] v2 data engine volume snapshot and revert 6137 - @shuo-wu @roger-ryao
• [FEATURE] Allow to set mount options for storageclass via values.yaml in helm chart 7351 - @ChanYiLin @chriscchien
• [UI][FEATURE] Longhorn snapshot space management 7522 - @scures
• [FEATURE] Flush on-the-fly IOs in the queue before snapshotting 5648 - @DamiaSan
• [FEATURE] Update base image of Longhorn components to BCI 15.5 6206 - @nitendra-suse
• [FEATURE] Customize MaxRecurringJobRetain 5713 - @mantissahz @chriscchien
• [FEATURE] Replica rebuild over SPDK 5216 - @shuo-wu @DamiaSan
• [FEATURE] Allow kubectl drain to stop manually attached volumes 6978 - @ChanYiLin @chriscchien
• [FEATURE] Single Node Disk affinity 3823 - @ejweber @roger-ryao
• [FEATURE] Storage network support for Multus v4.0 thick-plugin 5048 - @c3y1huang @chriscchien
• [FEATURE] Add disk status prometheus metrics 6858 - @c3y1huang @chriscchien
• [FEATURE] Add a brand new/empty bdev with WriteOnly mode to the RAID1 bdev 5865 - @DamiaSan
• [FEATURE] Manage replicas of RAID1 bdev 5420 - @DamiaSan
• [BUG] when volume degraded and rebuild, the filesystem in every mounted pod were read-only 1500 - @PhanLe1010
• [FEATURE] Add a script to identify the valid volumes to recover given s3 backup url and secret 1523 - @weizhe0422
• [FEATURE] Manage snapshot number or quota for storage capacity planning 6077 -
• [FEATURE] Pause IO when raid1 bdev snapshotting 5421 - @DamiaSan
• [FEATURE] Change the replica selector behavior so that an absent selector is able to select nodes without a TAG 4826 - @ChanYiLin @roger-ryao
• [FEATURE] Introduce write only mode in RAID1 5783 - @DamiaSan
• [FEATURE] Helm Chart make loglevel configurable 3655 - @mantissahz

Improvement

• [IMPROVEMENT] Have a clear message when reverting the parent of a volume-head snapshot for a v2 volume 7630 - @derekbit @shuo-wu
• [IMPROVEMENT] Enhance the code quality in the instance-manager instance and disk gRPC server methods. 7628 - @derekbit
• [TASK] Introduce upgradeVersionCheck to decide version upgrade enforcement 7539 - @mantissahz @chriscchien
• [IMPROVEMENT] Increase the hugepage size for spdk_tgt to 2GiB 7606 - @derekbit @chriscchien
• [IMPROVEMENT] Reject DR volume creation for v2 volume 7627 - @derekbit @roger-ryao
• [IMPROVEMENT] Do not use --force for dmsetup remove command 7615 -
• [IMPROVEMENT] Update nvme-cli to v2.7.1 in instance-manager pod 7609 - @derekbit
• [IMPROVEMENT] v2 volume snapshot supports UserCreated flag 7578 - @DamiaSan
• [IMPROVEMENT] Prevent from complains in spdk_tgt when deleting a v2 volume 7568 - @yangchiu @derekbit @roger-ryao
• [IMPROVEMENT] Expose actual size of a logical volume 5947 - @derekbit @shuo-wu @chriscchien @DamiaSan
• [IMPROVEMENT] UI backup restoration supports v1 and v2 Data Engine 6597 - @derekbit @scures @roger-ryao
• [IMPROVEMENT][UI] Display v2 volume actual size 7524 - @derekbit @chriscchien
• [IMPROVEMENT] Recreate instance manager pod for v2 volume when spdk_tgt is dead 7551 - @derekbit @chriscchien
• [IMPROVEMENT] Add reserve storage percentage of nodes setting in helm chart 5958 - @mantissahz @roger-ryao
• [IMPROVEMENT] Reconcile engine/replica instance state of v2 volume like v1 volume 7326 - @derekbit @chriscchien
• [IMPROVEMENT] Improve handling of 16TiB+ volumes with ext4 as the underlying file system 7423 - @mantissahz @chriscchien
• [IMPROVEMENT] Rename backendStoreDriver to dataEngin in instance-manager and associated components 7480 - @yangchiu @derekbit
• [IMPROVEMENT][UI] Validate volume creation according to the enabled data engines 7505 - @derekbit @chriscchien
• [IMPROVEMENT] Add guaranteed instanceManager CPU setting for v2 volume 7361 - @derekbit @roger-ryao
• [IMPROVEMENT] Automatically remount read-only RWO volume to read-write 6386 - @ChanYiLin @chriscchien
• [IMPROVEMENT] Support backup list if there is only v2-data-engine enabled 7486 - @derekbit @chriscchien
• [IMPROVEMENT] Upgrade CSI components to the latest patch release 7384 - @c3y1huang @roger-ryao
• [IMPROVEMENT] Add global setting for enable v1 or v2 volume support 7095 - @yangchiu @derekbit
• [IMPROVEMENT] Blindly stop raid bdev exposure before exposing it for V2 volume 7324 - @yangchiu @derekbit @roger-ryao
• [IMPROVEMENT] instance-managers for v1 and v2 volumes respectively 6984 - @yangchiu @derekbit
• [IMPROVEMENT] Reject volume.spec.image update for v2 volumes before introducing live upgrade 7446 - @derekbit @chriscchien
• [IMPROVEMENT] BackingImage should be compressed when downloading and use the name as filename instead of UUID 7295 - @ChanYiLin @chriscchien
• [IMPROVEMENT] Reject the creation of encrypted v2 volume in validating webhook 7404 - @derekbit @chriscchien
• [IMPROVEMENT] Longhorn-engine processes should refuse to serve requests not intended for them 5845 - @ejweber @chriscchien
• [IMPROVEMENT] Collect v2 Data Engine related info for the usage metrics 6033 - @c3y1huang @chriscchien
• [IMPROVEMENT] Review and simplify longhorn component image build 5911 - @ChanYiLin @chriscchien
• [IMPROVEMENT] Gracefully shut down spdk_tgt 7263 - @derekbit @chriscchien
• [IMPROVEMENT] Reject the last replica deletion if its volume.spec.deletionTimestamp is not set 7372 - @yangchiu @derekbit
• [IMPROVEMENT] add build script to generate gRPC related code more convenient 6973 - @Vicente-Cheng
• [IMPROVEMENT] Volumes: metrics for snapshots include (size and type: system vs user) 5869 - @c3y1huang @chriscchien
• [IMPROVEMENT] Upgrade support bundle kit version to v0.0.33 7277 - @c3y1huang
• [IMPROVEMENT] Upgrade CSI sidecar components version 6916 - @c3y1huang @roger-ryao
• [IMPROVEMENT] Have a setting to disable snapshot purge for maintenance purpose 7075 - @ejweber @roger-ryao
• [IMPROVEMENT] Don't crash the migration engine when kubelet restarts 7302 - @ejweber @chriscchien
• [IMPROVEMENT] deploy: driver deployer shouldn't cleanup previous deployment if Kubernetes version changes 5474 - @PhanLe1010 @chriscchien
• [IMPROVEMENT] Replace deprecated grpc.WithInsecure 7291 - @c3y1huang
• [IMPROVEMENT] Reconcile replica instance state after its associated block-type disk is ready 7338 - @derekbit
• [IMPROVEMENT] Allow deployment of Prometheus ServiceMonitor with the Longhorn helm chart 7041 - @mantissahz @chriscchien
• [IMPROVEMENT] Disable CGO in longhorn components if not used 7135 - @derekbit
• [IMPROVEMENT] Add test for longhorn-spdk-engine 6060 - @shuo-wu
• [IMPROVEMENT] Thread-safe SPDK JSON client 6106 - @shuo-wu
• [IMPROVEMENT] Bypass upgrade when installing a fresh setup 6988 - @mantissahz @roger-ryao
• [IMPROVEMENT] Upgrade support bundle kit version to v0.0.32 7152 - @c3y1huang @chriscchien
• [IMPROVEMENT] Support custom options for network filesystems for backup 6608 - @james-munson @roger-ryao
• [IMPROVEMENT] Global setting default-data-path supports block device 7234 - @derekbit @chriscchien
• [IMPROVEMENT] Clean up backup target in IM-R pod if the backup target setting is unset 5741 - @ChanYiLin @chriscchien
• [IMPROVEMENT] Improve log level for resource update failure able to reconcile again 6843 - @PhanLe1010 @nitendra-suse
• [IMPROVEMENT] Add missing volume settings to the default storage classs 6496 - @james-munson
• [IMPROVEMENT] High memory consumption of longhorn-manager pods since Longhorn v1.5 6936 - @derekbit @roger-ryao
• [IMPROVEMENT] Upgrade support bundle kit version to v0.0.29 6922 - @c3y1huang @chriscchien
• [IMPROVEMENT] Ensure Longhorn components Golang version upgrade to v1.21 and backport to v1.5.x, 1.4.x 7148 - @c3y1huang
• [IMPROVEMENT] Improve upgrade path and make it more solid 6294 - @PhanLe1010 @roger-ryao
• [IMPROVEMENT] Use nvme-cli in instance-manager pod instead 6798 - @derekbit @chriscchien
• [IMPROVEMENT] Add PVC namespace to longhorn_volume metrics 7077 - @mantissahz @roger-ryao @antoninferrand
• [IMPROVEMENT] Don't log about inability to change settings that didn't change. 6812 - @james-munson @roger-ryao
• [IMPROVEMENT] Consolidate the mounts in longhorn-manager and instance-manager 5883 - @ChanYiLin
• [IMPROVEMENT] Make the timeout value of a filesystem-based backup store configurable 5723 - @ChanYiLin
• [IMPROVEMENT] Unify logs with extra static info like module/method/function/line 5509 - @ChanYiLin @roger-ryao
• [IMPROVEMENT] Prevent Volume Provision if Related Backing Image Stuck in Ready-For-Trasfer State 6615 - @ChanYiLin @roger-ryao
• [IMPROVEMENT] Remove dummy services of each CSI sidecar if not required 6581 - @ejweber @roger-ryao
• [IMPROVEMENT] Old kernel such as 3.10.0 set provisioning_mode to wrong value (writesame_16, disabled, full, ...) but not the correct value (unmap) so the trim feature doesn't work 6854 - @PhanLe1010 @chriscchien
• [IMPROVEMENT] Support both NFS hard and soft with custom timeo and retrans options for RWX volumes 6655 - @derekbit @roger-ryao
• [IMPROVEMENT] Prevent unexpected engine creation 6682 - @PhanLe1010 @ejweber @roger-ryao
• [IMPROVEMENT] Add nvmecli to IM image 6517 - @c3y1huang
• [IMPROVEMENT] Add pvc name to longhorn_volume metrics 5297 - @c3y1huang @nitendra-suse
• [IMPROVEMENT] Replace engineImage field in CRDs with image 6647 - @derekbit @chriscchien
• [IMPROVEMENT] Fix scheduling flooding logs 6019 - @ChanYiLin @roger-ryao
• [IMPROVEMENT] Avoid the accident deletion of longhorn settings 4984 - @ejweber @roger-ryao
• [IMPROVEMENT] Restoring a backup to create a volume should keep the original storage class 4063 - @PhanLe1010
• [IMPROVEMENT] UI: making batch deletion dialog more readable 4080 - @smallteeths
• [IMPROVEMENT] Upgrade Longhorn upgrade-responder server and build new Grafana dashboard 6368 - @PhanLe1010
• [IMPROVEMENT] Consider adding owner reference Backup/BackupVolume CR 5896 - @ChanYiLin
• [IMPROVEMENT] Include /var/log/messages during the support-bundle syslog collection 6544 - @c3y1huang @roger-ryao
• [IMPROVEMENT] UI Volume detail page still shows Block Device when spec.disableFrontend is true 6167 - @smallteeths @chriscchien
• [IMPROVEMENT] Remove Longhorn engine path mismatch log 3786 - @c3y1huang @roger-ryao
• [IMPROVEMENT] Provide more information for volume scheduling failure 6461 - @smallteeths @chriscchien
• [IMPROVEMENT] Implement/fix the unit tests of Volume Attachment and volume controller 6005 - @PhanLe1010 @roger-ryao
• [QUESTION] Repetetive warnings and errors in a new longhorn setup 6257 - @derekbit @c3y1huang @roger-ryao
• [IMPROVEMENT] Make environment check script recognize iscsid.socket enable instead of iscsid.server only 5380 - @derekbit @roger-ryao

Bug

• [BUG] Backing Image Data Inconsistency if it's Exported from a Backing Image Backed Volume 6899 - @ChanYiLin
• [BUG] Deleting instance-manager during restoring a v2 volume, the volume stuck in detaching state 7581 - @derekbit @chriscchien @roger-ryao
• [BUG] replica not rebuild in v1.6.0-dev if engine image is v1.4.x 7631 - @mantissahz
• [BUG] Deleting instance-manager pod causes v2 volume stuck in attaching/detaching loop 7579 - @derekbit @roger-ryao
• [BUG] After some v2 volume operations, v2 instance manager on a specific node somehow doesn't work 7608 - @yangchiu @derekbit
• [BUG] Inconsistent behavior of snapshot list between v1 and v2 volume 7622 - @yangchiu @derekbit
• [BUG] After v2 volume offline rebuilding, re-attached volume remains degraded 7574 - @yangchiu @shuo-wu
• [BUG] Fix and improve the offline rebuilding after introducing the SPDK snapshot feature 7596 - @shuo-wu @chriscchien
• [BUG] Remove v2 volume rebuild snapshot could cause volume stuck in detaching/faulted state 7573 - @shuo-wu
• [BUG] Backup volume attachment tickets might not be cleaned up after completion. 6654 - @james-munson @chriscchien
• [BUG] Unable to list backups when backuptarget resource is picked up by a cordoned node 7619 - @c3y1huang
• [BUG] Correct the naming of v2 volume snapshot created after backup restoration 7577 - @derekbit @chriscchien
• [BUG] Warning events are being spammed by Longhorn - CRD 7290 - @m-ildefons @roger-ryao
• [BUG] Randomly failed to create volume with backing image 7543 - @yangchiu @ChanYiLin
• [BUG] v2 volume becomes faulted and detached after deleting one replica during full restoration 7597 - @derekbit @chriscchien
• [BUG] Creating volume randomly failed: failed to find a node that is ready and has the default engine image 7413 - @yangchiu @PhanLe1010
• [BUG] Delete error backup could cause v2 volume stuck in detaching/faulted state 7575 - @derekbit @roger-ryao
• [BUG] Restore v2 volume stuck in detaching/faulted state if the backup is corrupted 7583 - @derekbit @chriscchien
• [BUG] After upgrade to master-head, existing volume won't rebuild replica if one deleted, and the volume keeps healthy instead of degraded 7555 - @FrankYang0529 @yangchiu @derekbit
• [BUG] test case test_metric_longhorn_snapshot_actual_size_bytes failed 7532 - @yangchiu @c3y1huang
• [BUG] Delete the backup during restoring a v2 volume from the backup, the restore volume will be detached and faulted 7584 - @derekbit
• [BUG] Delete v2 volume snapshot make volume detaching faulted 7585 - @derekbit
• [BUG] Fix the failure of test_basic.py:: test_volume_scheduling_failure for v2 volumes 7570 - @derekbit @chriscchien
• [BUG] Fix using deprecated option of blockdev command in go-spdk-helper 7567 - @derekbit
• [BUG] Delete kubernetes node did not remove node.longhorn.io 7475 - @ejweber @chriscchien
• [BUG] Failed to check_volume_data after volume engine upgrade/migration 7396 - @PhanLe1010 @james-munson @roger-ryao
• [BUG] volume engine failed to live upgrade 7548 - @derekbit @chriscchien
• [BUG] test case test_engine_image_not_fully_deployed_perform_auto_upgrade_engine failed 7540 - @c3y1huang
• [BUG] Failed RWX mount due to connection timeout still happening 7301 - @james-munson
• [BUG] V2 volume is attached to a node first, the V1 volume will fails to attach. 7511 - @c3y1huang @roger-ryao
• [BUG] v2 volume always displays engine upgrade available on UI 7489 - @scures
• [BUG] Create volume(v1) faulted 7536 - @FrankYang0529 @chriscchien
• [BUG] Running new e2e test pipeline always failed with error not ready 7484 - @PhanLe1010 @chriscchien
• [BUG] Persistent volume is not ready for workloads 6776 - @james-munson @roger-ryao
• [BUG] Unable to create snapshot: cannot get engine client because it isn't deployed 7438 - @yangchiu @PhanLe1010
• [BUG] Deadlock for RWX volume if an error occurs in its share-manager pod 7183 - @derekbit @chriscchien
• [BUG] Volume conditions are not represented in the UI for v1.4.x and newer 7241 - @m-ildefons @chriscchien
• [BUG] backingimage download server error 7288 - @scures @roger-ryao
• [BUG] CSI components CrashLoopBackOff, failed to connect to unix://csi/csi.sock after cluster restart 7116 - @yangchiu @ejweber
• [BUG] Kubelet cannot finish terminating a pod that uses a PVC with volumeMode: Block when restarting the node 6919 - @PhanLe1010 @chriscchien
• [BUG] Test case test_node_default_disk_labeled failed 7385 - @derekbit @roger-ryao
• [BUG] Helm2 install error: 'lookup' function not defined in validate-psp-install.yaml 6318 - @innobead @roger-ryao
• [BUG] Cleint in go-spdk-helper is stuck after encoutering IO timeout 7395 - @derekbit @chriscchien
• [BUG] DataEngineV2 Unable to attach a PV to a pod in the newer kernel 7190 - @yangchiu @derekbit
• [BUG] orphaned pod pod_id found, but error not a directory occurred when trying to remove the volumes dir 3207 - @weizhe0422 @roger-ryao
• [BUG] Download backing image failed with HTTP 502 error if Storage Network configured 7236 - @ChanYiLin @roger-ryao
• [BUG] During volume live engine upgrade, delete replica with old engine image will make volume degraded forever 7012 - @PhanLe1010 @chriscchien
• [BUG] A race after a node reboot leads to I/O errors with migratable volumes 6961 - @yangchiu @ejweber
• [BUG] Metric totalVolumeSize and totalVolumeActualSize incorrect due to v2 volume counts 7380 - @c3y1huang @chriscchien
• [BUG] Longhorn-manager does not deploy CSI driver when integrated with linkerd service mesh 3809 - @mantissahz @chriscchien
• [BUG] Test case test_node_eviction failed 7210 - @ejweber @roger-ryao
• [BUG] Cannot add block-type disk to node resource due to timeout error 7253 - @yangchiu @shuo-wu
• [BUG] multiple "for-cloning-volume" snapshots created after cloning volume 5835 - @PhanLe1010 @chriscchien
• [BUG] Volume has 2 active engines at the same time that blocks the volume controller reconciliation loop 4827 - @PhanLe1010 @chriscchien @roger-ryao
• [BUG] Volume UI displays only the last backup when using the recurring job 2997 - @mantissahz @chriscchien @roger-ryao
• [BUG] Volume gets stuck in an unknown state forever if created in an engine not fully deployed environment 6131 - @yangchiu @PhanLe1010
• [BUG] Continuously auto-balancing replicas when zone does not have enough space 6671 - @yangchiu @c3y1huang @roger-ryao
• [BUG] backing-image-manager- hostPath selection exception 7062 - @ChanYiLin @chriscchien
• [BUG] GET error for volume attachment on node reboot 4188 - @PhanLe1010
• [BUG] Errors found by static checker in volume controller 7009 - @m-ildefons
• [BUG] Enabling replica-auto-balance tries to replicate to disabled nodes causing lots of errors in the logs and in the UI 6508 - @c3y1huang @chriscchien
• [BUG] Confusing logging when trying to attach a new volume with no scheduled replicas 7244 - @ejweber @chriscchien
• [BUG] Supported minimum K8s version not correct 7196 - @c3y1huang
• [BUG] allow-collecting-longhorn-usage-metrics setting is missing from chart settings 7050 - @ChanYiLin @yardenshoham @roger-ryao
• [BUG] Longhorn storage network is incompatible with Multus version above v4.0.0 6953 - @c3y1huang @chriscchien
• [BUG] The archived docs page is broken 7222 - @innobead
• [BUG] Longhorn gives 500 error when trying to provision a volume created using a snapshot. 4785 - @ChanYiLin
• [IMPROVEMENT] Optimize the resource cache to prevent high memory usage in longhorn-manager 6954 - @derekbit @nitendra-suse
• [DOC] longhorn-csi-plugin stuck in CrashLoopBackOff after system crash (SELinux related) 5348 - @ejweber
• [BUG] Cannot detach the restored volume when there is a node goes down during restoring 2103 - @ejweber @chriscchien
• [BUG] Failing to mount encrypted volumes 7033 - @mantissahz @chriscchien
• [BUG] The instance manager with state unknown will be cleaned up in the split-brain case 6479 - @shuo-wu @chriscchien
• [BUG] Orphan snapshot attachment tickets prevent volume from detaching 6652 - @ejweber @chriscchien
• [BUG] Test case test_system_backup_and_restore failed 7143 - @ChanYiLin @roger-ryao
• [BUG] missing description in support-bundle metadata.yaml 6997 - @c3y1huang @roger-ryao
• [BUG] Cannot mount XFS PV 7140 - @PhanLe1010 @roger-ryao
• [BUG] Volume encryption doesn't work on Amazon Linux 2 5944 - @derekbit @chriscchien
• [BUG] Test case test_csi_minimal_volume_size failed 7170 - @roger-ryao
• [BUG] Deleting a PVC bound to a CSI PV, will delete associated volume and the CSI PV in result. 7172 -
• [BUG] Relax S3 client retry intervals, for throttled requests 2810 - @mantissahz @chriscchien
• [BUG] supportbundle/kubelet.log empty in k3s environment 7121 - @c3y1huang @chriscchien
• [BUG] Failing to mount encrypted volumes v1.5.2 7045 - @derekbit @nitendra-suse
• [BUG] Invalid volume name containing less-than sign 7092 -
• [BUG] Somehow the Rebuilding field inside volume.meta is set to true when one replica only, causing the volume into attaching/detaching loop 6626 - @c3y1huang @nitendra-suse
• [BUG] [longhorn-engine] [s390x] intermittent fail pipeline on build step 6975 - @Anarkis
• [BUG] Longhorn Read-Only setting can be modified 5989 - @mantissahz @roger-ryao
• [BUG] UI: All components handle window resizing events incorrectly 7036 - @votdev
• [BUG] UI: The action menu handler should stop event propagation 7032 - @votdev
• [BUG] longhorn manager isn't annotated with iam.amazonaws.com/role 6947 - @mantissahz @chriscchien
• [BUG] invalid memory address or nil pointer dereference in BackupVolumeController 6998 - @mantissahz @roger-ryao
• [BUG] Longhorn manager pods in 1.5.1 consuming 20GB+ RAM and 3-4 vCPUs 6866 - @derekbit @shuo-wu
• [BUG] MountVolume.MountDevice failed for volume Output: mount.nfs: Protocol not supported 6887 - @derekbit
• [BUG] High CPU usage on one node. 6578 - @derekbit @chriscchien
• [BUG] Set a invalid backup target when backup in progress will cause backup never finish 6491 - @ChanYiLin @chriscchien
• [BUG] duplicate MIME type "text/html" in /var/config/nginx/nginx.conf 7002 - @votdev
• [BUG] After crashed engine process, volume stuck in Unknown state 6699 - @ChanYiLin @nitendra-suse
• [BUG] Longhorn Instance Manager Memory leak 6481 - @james-munson @chriscchien
• [BUG] Two active engine when volume migrating 6642 - @PhanLe1010 @chriscchien
• [BUG] Button "Take Snapshot" and "Create Backup" grayed out. 6841 - @votdev
• [BUG] test_exporting_backing_image_from_volume failed in HDD test 4471 - @ChanYiLin
• [BUG] Environment Check Script Fails To Perform All Checks 5653 - @PhanLe1010 @roger-ryao
• [BUG] Suggest to provide more intuitive error message for backing image with unsupported file format 3294 - @ChanYiLin
• [BUG] Volumes failing to mount because of engine upgradedReplicaAddressMap reference 6762 - @PhanLe1010 @chriscchien
• [BUG] Expansion error still occurs in 1.4.2 6078 - @ejweber
• [BUG] Unable to add a block-type disk with a new name 6849 - @derekbit @chriscchien
• [BUG] IO error occurs when detaching RWX volume 6829 - @derekbit @chriscchien
• [BUG] DR volume failed when synchronizing the incremental backup 6750 - @mantissahz @chriscchien
• [BUG] Salvage failing in attaching and detaching loop, another pod is attached with health unknown 6662 - @james-munson
• [BUG] Replica should reported stopped if the instance manager cannot be found 1118 - @ejweber @chriscchien
• [BUG] Cannot detach volume 6688 - @james-munson
• [BUG] 1.5.0: AttachVolume.Attach failed for volume, the volume is currently attached to different node 6287 - @yangchiu @derekbit
• [BUG] Helm installation with privateRegistry.registryUrl set doesn't work 3057 - @PhanLe1010 @chriscchien
• [BUG] Node shows all replicas as stopped 5582 -
• [BUG] Fix errors in questions.yaml 6392 - @james-munson @chriscchien
• [BUG] cifs backup mount paths with dollar sign are not allowed 6660 - @derekbit @roger-ryao
• [BUG] Failed Statefulset Pod Creation with RWX Workload on Longhorn v1.3.3 and SLES 15 SP5 6494 - @ejweber @roger-ryao
• [BUG] Failure to update backup status leads to infinite reconciliation 6358 - @ejweber @chriscchien
• [BUG] longhorn installation randomly failed on sles 15-sp5 due to longhorn manager CrashLoopBackOff 6504 - @ejweber @chriscchien
• [BUG] Can't delete volumesnapshot if backup target not set 4979 - @ejweber @chriscchien
• [BUG] Share manager pod will stay in IO error when the volume becomes read only 5961 - @ChanYiLin @roger-ryao
• [BUG] Invalid Helm default setting value in Longhorn v1.5.1 6620 - @c3y1huang
• [BUG] SettingNameSnapshotDataIntegrityCronJob should be sent as boolean value 6410 - @c3y1huang @roger-ryao
• [BUG] Permission denied when starting longhorn-ui container 6430 - @mantissahz @chriscchien
• [BUG] Longhorn manager crashed during backing image 100gb volume export 5209 - @ChanYiLin @chriscchien
• [BUG] Removed IM CPU request settings still exists and new IM CPU request missed from chart settings 6465 - @c3y1huang @chriscchien
• [BUG] Error during backup process will be removed quickly without user knowing 1249 - @mantissahz @chriscchien
• [BUG] PV using v2 engine cannot attach 6441 - @derekbit @chriscchien @nitendra-suse
• [BUG] Backup Job returns "Completed" despite running into errors 4255 - @mantissahz @chriscchien
• [BUG] 1.5.0 Upgrade: Longhorn conversion webhook server fails 6259 - @derekbit @roger-ryao
• [BUG] Webhook is never called for BackingImageManager 6328 - @ejweber @chriscchien
• [BUG] Error message not getting cleaned up on switching the backupstore 2944 - @mantissahz
• [BUG] Unable to list backup from a local backupstore in RKE2 CIS-1.23 environment 6342 - @mantissahz
• [BUG] test case test_inc_restoration_with_multiple_rebuild_and_expansion randomly failed 5496 - @mantissahz
• [BUG] disk monitor cannot recognize disks if disk paths are somehow changed after reboot 6125 - @yangchiu @derekbit
• [BUG] Can not delete type=bi VolumeSnapshot if related backing image not exist 6266 - @ChanYiLin @chriscchien
• [BUG] Race leaves snapshot CRs that cannot be deleted 6298 - @yangchiu @PhanLe1010 @ejweber
• [BUG] test case test_setting_priority_class failed in master and v1.5.x 6319 - @derekbit @chriscchien
• [BUG] Upgrade to 1.5.0 failed: validator.longhorn.io denied the request if having orphan resources 6246 - @derekbit @roger-ryao
• [BUG] test_recovery_from_im_deletion cannot pass in particular environment 6171 - @ejweber
• [BUG] Minior typo in a lhv yaml stop entire cluster from working 2423 - @mantissahz
• [BUG] Longhorn Manager Pods CrashLoop after upgrade from 1.4.0 to 1.5.0 while backing up volumes 6264 - @ChanYiLin @roger-ryao
• [BUG] Unable to receive support bundle from UI when it's large (400MB+) 6256 - @c3y1huang @chriscchien
• [BUG] Live upgrade stuck if the same volume name backup exists in the backup store 3403 - @ChanYiLin @chriscchien
• [BUG] RWX volume unable to mount in rke2 cluster. 2659 - @derekbit
• [BUG] Test case test_inc_restoration_with_multiple_rebuild_and_expansion failed on SLES 6089 - @c3y1huang
• [BUG] Unexpected "Failed" volume workload pod status on k8s v1.27 6097 - @derekbit
• [BUG] test_snapshot_hash_detect_corruption_in_global_enabled_mode failed due to not proper cron job value 5394 - @derekbit
• [BUG] Instance manager may not update instance status for a minute after starting 5809 - @ejweber @chriscchien

Performance

• [TASK] Add 1.5 performance benchmark to performance benchmark WIKI page 6203 - @derekbit
• [FEATURE] Increase read bandwidth of SPDK volume from all downstream replicas 5759 - @derekbit @chriscchien

Benchmark

• [TASK] Performance benchmark when using Storage network with or without a customized MTU setting 6996 - @yangchiu

Misc

• [TASK] Investigate if we can automate the Rancher chart release 7023 - @mantissahz @PhanLe1010
• [TASK] Update CLIAPIVersion in longhorn-manager 7588 - @FrankYang0529 @roger-ryao
• [TASK] Security vulnerabilities in docker images 7523 - @c3y1huang @roger-ryao
• [BUG] Test case test_volume_reattach_after_engine_sigkill failed 6751 - @c3y1huang
• [TASK][UI] v2 volume does not support engine image upgrade 7445 - @chriscchien @scures @roger-ryao
• [DOC] Add missing descriptions for Helm 7485 - @mantissahz
• [TASK] Update protoc to v24.3 6666 - @FrankYang0529
• [FEATURE] Enable resource profiling for IM 6377 - @derekbit @roger-ryao
• [TASK] Synchronize version of CSI components in longhorn/longhorn and longhorn/longhorn-manager 7377 - @c3y1huang @roger-ryao
• [TASK] Upgrade csi-snapshotter to mitigate rapid retry bug 6506 - @ejweber
• [TASK] Remove engine image dependency of v2 volumes 7157 - @derekbit
• [DOC] Fix erronous value for default StorageMinimalAvailablePercentage setting. 7342 - @james-munson
• [DOC] FS Trim for RWX is supported, but docs are out of date. 6733 - @james-munson
• [REFACTOR] Node Controller Unit Tests 7332 - @m-ildefons
• [DOC] Create document for argocd 7360 - @yangchiu
• [DOC] Create document for flux 7335 - @yangchiu
• [DOC] Create document for fleet 7331 - @yangchiu
• [REFACTOR] Abstract the disk/lvol file operations in backupstore 6576 - @derekbit @chriscchien
• [TASK] Remove namespaceOverride from the chart values 6527 - @mantissahz @roger-ryao
• [TASK] Implement xattr get and set operations on SPDK logical volumes (lvol) 6604 - @derekbit
• [DOC] Stress using object store as best practice for backups. 6773 - @james-munson
• [TASK] Investigate and update the Rancher chart annotation requirements 7024 - @PhanLe1010 @chriscchien
• [TEST] tests.test_basic.test_aws_iam_role_arn failed on SLES on 1.4.x (AMD) 6912 - @yangchiu
• [DOC] Run fsck.ext4 on newer Longhorn volume from older Linux distro 6859 - @ejweber @roger-ryao
• [TASK] Investigate how SPDK handle data integration of lvol snapshots 5578 - @DamiaSan
• [TASK] Move common functions for backup to backupstore lib 6514 - @derekbit
• [BUG] Cannot mount ReadWriteMany (RWX) volume in pod 7208 -
• [TASK] Investigate SELinux enabled with Longhorn 6074 - @yangchiu @ejweber
• [IMPROVEMENT] List of Longhorn Helm Chart Flags 5455 - @ChanYiLin
• [REFACTOR] UI: Disable Delete menu for default engine image 7029 - @votdev
• [TASK][UI] Replace spec.engineImage field in volume, engine and replica CRDs with spec.image 6685 - @votdev
• [TEST] tests.test_basic.test_default_storage_class_syncup failed on SLES on 1.4.x (ARM) 6913 - @yangchiu
• [EPIC] Side effects of increasing resync period in informer's event handlers 3629 - @PhanLe1010
• [TASK] The development branch should reference to the head images in longhorn-image.txt 6737 - @c3y1huang @chriscchien
• [TASK] Build dependent host libraries (liblonghorn & tgt) based on rpm for BCI adoption 3854 - @nitendra-suse
• [DOC] Describe how PDB used for instance manager to avoid data loss 4838 - @PhanLe1010
• [TASK] Create a CIFS backup store example in longhorn repo 6530 - @chriscchien
• [DOC] Explanation of storage class parameters 4776 - @james-munson @roger-ryao
• [DOC] Create a KB for high space consumption issue guideline 6592 - @shuo-wu
• [DOC] Create a KB for incorrect replica expansion 6391 - @ejweber
• [DOC] deploy/longhorn.yaml out of date - causes all longhorn-manager instances to crash-loop 6428 - @c3y1huang
• [REFACTORING] Move adding finalizer of resources to mutation webhooks as volume/engine/replica 4872 - @ejweber @chriscchien
• [DOC] Improve the documentation about troubleshooting and worse/best practice of operations 3164 - @c3y1huang
• [TASK] Update or remove out-of-date cleanup script 6316 - @james-munson
• [DOC] v1.5.0 additional outgoing firewall ports need to be opened 9501 9502 9503 6317 - @ChanYiLin @chriscchien
• [TASK] Check and update the networking doc & example YAMLs 5651 - @yangchiu @shuo-wu
• [TEST][FEATURE] Auto Trim via recurring job 5528 - @c3y1huang

Changes in this release

longhorn/longhorn-ui

• release: update version file for v1.6.0-rc1 00a7f305 by @innobead
• restore: add data engine option 95407604 by @derekbit
• Disables Upgrade action & icon for V2 volumes 63663961 by @scures
• v2 volume: display v2 volume actual size 78130238 by @derekbit
• volume: validate volume creation according to the enabled data engines be240854 by @derekbit

longhorn/longhorn-share-manager

• release: update version file for v1.6.0-rc1 75af5e10 by @innobead
• Use nfs-ganesha v5 branch. b2f876fd by @james-munson
• fix(cve): update kubernetes to v1.28.5 042929cd by @c3y1huang
• Create PULL_REQUEST_TEMPLATE.md e3144b44 by @innobead
• chore(vendor): update go-common-libs cbfb9095 by @c3y1huang

longhorn/longhorn-engine

• release: update version file for v1.6.0-rc1 5e1337d9 by @innobead
• backup/restore: update arguments due to the changes in apis deb8b18a by @derekbit
• vendor: update backupstore 0490dee7 by @derekbit
• vendor: bump go-common-lib for missing pprof 3848d7dd by @Vicente-Cheng
• fix(cve): update grpc_health_probe to v0.4.24 54457db0 by @c3y1huang
• profiler: add profiler command b41e8109 by @Vicente-Cheng
• vendor: update go-common-libs for profiler 358264da by @Vicente-Cheng
• Update Dockerfile.dapper 662b1614
• ci: make docker cache miss 339f5c10 by @FrankYang0529
• feat: bump CLIAPIVersion to 10 921d926b by @FrankYang0529
• Create PULL_REQUEST_TEMPLATE.md f86f4a84 by @innobead
• test: exit when there is first failure 12e92e84 by @FrankYang0529
• feat: support snapshot max count and size cc796ff9 by @FrankYang0529
• feat(proto): support snapshot max count and size 29ee44dd by @FrankYang0529
• chore(vendor): update go-common-libs 5688735e by @c3y1huang
• ci: remove unused folder 53562fe8 by @FrankYang0529
• fix: lint babd8cd9 by @FrankYang0529
• test: fix engine/replica identity error check 4ec45f54 by @innobead
• test: fix direct read/write start page calculation ae2aeda1 by @innobead
• feat: update longhorn-instance-manager in Dockerfile.dapper 563531de by @FrankYang0529
• fix: integration test fe7a5656 by @FrankYang0529
• feat(proto): upgrade to v24.3 c522c5c1 by @FrankYang0529
• feat: upgrade packages version 769ab488 by @FrankYang0529
• vendor: update backupstore 5efed2b8 by @derekbit

longhorn/longhorn

• release: v1.6.0-rc1 ca3d73a2 by @innobead
• fix: sync crd from longhorn-manager repo dc1179a9 by @innobead
• Update bug.md 2a13f1ea by @innobead
• fix(upgrade): skip to check engine upgrade path d918ad3f by @mantissahz
• setting: add v2-data-engine-hugepage-limit b9ed3e40 by @derekbit
• Incease default hugepage size for v2 data engine to 2GiB 62074d77 by @derekbit
• Update feature.md 1d755966 by @innobead
• Update improvement.md fc16697c by @innobead
• fix(upgrade): skip upgrade path check b93eebc1 by @mantissahz
• Allow to set NFS mount options for storageclass via values.yaml in helm chart 84ea854e by @yardenshoham
• Update infra.md 96a09957 by @innobead
• Update infra.md cf93f9a3 by @innobead
• Update feature.md 1187e0e4 by @innobead
• Update ci.md 0c96083b by @innobead
• Update bug.md f5dd46a7 by @innobead
• Update bug.md eabe7bcc by @innobead
• Update bug.md 9cc4b1e9 by @innobead
• Create PULL_REQUEST_TEMPLATE.md b58c8b8e by @innobead
• chart: fix missing fields in engine 3ac2efa1 by @derekbit
• lep: add Support Backup and Restore For Volumes with V2 Data Engine dd716339 by @derekbit
• feat(lep): add snapshot space management 201cd37a by @FrankYang0529
• feat(chart): add snapshotMaxCount and snapshotMaxSize 297f3152 by @FrankYang0529
• doc: add descripions in chart/value.yaml b2ad63f0 by @mantissahz
• doc(backupbackingimage): add example yaml of backingimage backup and restore 049053f0 by @ChanYiLin
• feat(backingimage): backup backing image support 27b67280 by @ChanYiLin
• chart: rename backendStoreDriver to dataEngine 38fa709e by @derekbit

longhorn/longhorn-manager

• release: update version file for v1.6.0-rc1 01603de0 by @innobead
• feat: add updateSnapshotMaxCount and updateSnapshotMaxSize actions 98189bfa by @FrankYang0529
• feat(setting): lazy update for volume-related settings 273b6c92 by @mantissahz
• vendor: update version 01ee8388 by @derekbit
• fix: engine upgrade enforcement should not prevent existing volumes operations 3a118a88 by @innobead
• fix(upgrade): skip to check engine upgrade path 30db3a71 by @mantissahz
• Reject DR volume creation for v2 volume eeda02be by @derekbit
• feat: add snapshotMaxCount and snapshotMaxSize to Volume API d20213c9 by @FrankYang0529
• vendor: update dependencies 59194602 by @derekbit
• Incease default hugepage size for v2 data engine to 2GiB 1cb5e252 by @derekbit
• 6654 - Move deletion of backup volume attachment ticket. 9067c7c9 by @james-munson
• fix: replace backend store driver with data engine c2c799d3 by @FrankYang0529
• v2 volume: fix the conflict of offline rebuilding and full restoration 2ac44e55 by @derekbit
• fix: refactor naming 7a56d710 by @innobead
• fix(upgrade): skip upgrade path check 6c30f953 by @mantissahz
• fix(upgrade): add v140 to v15x upgrade path back f78ad148 by @mantissahz
• feat: bump longhorn-engine 2a256a33 by @FrankYang0529
• backup controller: fix log message 2931e5b7 by @derekbit
• v2 volume: prevent from errors in spdk_tgt when deleting a volume 1a206d90 by @derekbit
• Fix errors found by staticcheck e5f7071d by @derekbit
• CheckDataEngineImageReadiness() should handle v2 data engine as well. fa1fd028 by @derekbit
• fix(cve): update kubernetes to v1.28.5 b8ba9013 by @c3y1huang
• fix: add condition for updating snapshot attritube to engine a601b9b1 by @FrankYang0529
• fix: add condition for updating snapshot attritube to engine ac6bf684 by @FrankYang0529
• refactor(webhook): break for early exit in finalizer check ddd5ae7d by @c3y1huang
• fix(util): resource name should only contain lower case 32525238 by @ChanYiLin
• ci: node controller tests for node conditions cb2abae4 by @m-ildefons
• ci: events on node condition tests df29e5de by @m-ildefons
• ci: add tests for unknown node conditions ed7a763e by @m-ildefons
• instance manager: update liveness probe 94a659f9 by @derekbit
• Allow deletion of node finalizer without passing checks e5327399 by @ejweber
• Fix nil pointer dereference in shouldEvictReplica 3921faf3 by @ejweber
• Create PULL_REQUEST_TEMPLATE.md 1ccf63d5 by @innobead
• feat(backingimage): backup backing image support 970ba447 by @ChanYiLin
• vendor: update dependencies 2e69c0b6 by @derekbit
• spdk/snapshot: remove spdk creation rejection in validating webhook f563ecbb by @derekbit
• feat: support snapshot max count and size 30befe51 by @FrankYang0529
• feat(crd): add SnapshotMaxCount and SnapshotMaxSize 0ca90e39 by @FrankYang0529
• feat(go.mod): update longhorn-engine and longhorn-instance-manager 80cf13fc by @FrankYang0529
• Support instance-manager pod for v2 data engine on selected nodes 91bc9cb7 by @derekbit
• chore(vendor): update go-common-libs 420b7798 by @c3y1huang
• feat: add DataSourceTypeExportFromVolumeParameterFileSyncHTTPClientTimeout b2132e74 by @FrankYang0529
• feat: upgrade packages version de40f934 by @FrankYang0529
• feat(instance status): reset instance error status when it is not running b63565de by @ChanYiLin
• crd: fix typo 9ef31e1e by @derekbit
• crd: add description for deprecated backendStoreDriver field 86b573ca by @derekbit
• Replace backendStoreDriver with dataEngine 655d52e8 by @derekbit
• vendor: update dependencies 29ab93c3 by @derekbit
• controller: use available data engine for getting backup target client a012b4c0 by @derekbit

longhorn/backing-image-manager

• release: update version file for v1.6.0-rc1 cf602da1 by @innobead
• Create PULL_REQUEST_TEMPLATE.md 28b787c0 by @innobead
• fix: fix golint issue need to catch return value 7f8aea1e by @ChanYiLin
• feat(backup): backing image backup support 70dd4a80 by @ChanYiLin

longhorn/longhorn-instance-manager

• release: update version file for v1.6.0-rc1 c14a4050 by @innobead
• disk grpc: refactor operations 3519c624 by @derekbit
• proxy: unify names of variables a0ce1c09 by @derekbit
• app: update arguments a4b651fe by @derekbit
• instance grpc: refactor operations 0f9d7ccd by @derekbit
• proxy: add volume-head to snapshot list 2903e519 by @derekbit
• vendor: update dependencies f5a00b06 by @derekbit
• app: update arguments due to the changes in apis 757b9e62 by @derekbit
• Do not use --force for dmsetup remove command ae1b763c by @derekbit
• vendor: update dependencies dffebbb1 by @derekbit
• vendor: update dependencies 25de8d4c by @derekbit
• Dockerfile: update nvme-cli to v2.7.1 fa0591d0 by @derekbit
• Make the best effort to clean up stale NVMe and device mapper devices 1a4d0c99 by @derekbit
• fix: replace backend store driver with data engine 8065e625 by @FrankYang0529
• vendor: update longhorn-spdk-engine 117db878 by @derekbit
• vendor: update longhorn-spdk-engine 385bc496 by @derekbit
• vendor: update dependencies 82446cbd by @derekbit
• fix(cve): update grpc_health_probe to v0.4.24 cf2b4d5c by @c3y1huang
• feat: bump InstanceManagerAPIVersion to 5 f8573fc6 by @FrankYang0529
• proto: regenerate files using correct versions 2502c579 by @derekbit
• Fix bug instance manager pod crashed after upgrading engine image 0474250e by @PhanLe1010
• proxy: update ProxyOps 304418ff by @derekbit
• vendor: update longhorn-spdk-engine 8e3abaa7 by @derekbit
• Dockerfile: add netcat package b619d880 by @derekbit
• Create PULL_REQUEST_TEMPLATE.md aa848a43 by @innobead
• vendor: update dependencies ce7a9dec by @derekbit
• vendor: update dependencies ab68378c by @derekbit
• v2 volume: expose snapshot actual size 5004f3bb by @derekbit
• v2 volume: add SnapshotRevert proxy function c005ae87 by @derekbit
• v2 volume: add SnapshotRemove proxy function 798b43da by @derekbit
• Dockerfile: update spdk to 72b7762674cc35262086b185a455a8a30f46432d 6014434d by @derekbit
• Dockerfile: remove ununsed package installation 8e28cf91 by @derekbit
• v2 volume: support backup and restore 82530a79 by @derekbit
• proto: add BackupRestoreFinish method 2ab63bed by @derekbit
• lint: supress suppress SA1019 6539eb6f by @derekbit
• proxy: remove redundant codes 0c01fe39 by @derekbit
• feat(proxy): support snapshot max count and size e09a1a82 by @FrankYang0529
• feat(proto): support snapshot max count and size 31144053 by @FrankYang0529
• feat(go.mod): update longhorn-engine cb926386 by @FrankYang0529
• chore(vendor): update go-common-libs 48ff2307 by @c3y1huang
• feat: sync latest proto from longhorn-engine 86f0f958 by @FrankYang0529
• feat: update proto cbade98f by @FrankYang0529
• Replace backendStoreDriver with dataEngine 60067ae0 by @derekbit
• vendor: update backupstore and longhorn-engine 834489e1 by @derekbit
• proto: replace backendStoreDriver with dataEngine edd76f2d by @derekbit

Contributors

• @Anarkis
• @ArthurVardevanyan
• @ChanYiLin
• @DamiaSan
• @FrankYang0529
• @PhanLe1010
• @Vicente-Cheng
• @antoninferrand
• @c3y1huang
• @chriscchien
• @derekbit
• @ejweber
• @innobead
• @james-munson
• @m-ildefons
• @mantissahz
• @nitendra-suse
• @roger-ryao
• @scures
• @shuo-wu
• @smallteeths
• @votdev
• @weizhe0422
• @yangchiu
• @yardenshoham