Releases: cloud-hypervisor/cloud-hypervisor
v29.0
This release has been tracked in our roadmap
project as iteration
v29.0. The following user visible changes have been made:
Release Binary Supports Both MSHV and KVM
On x86-64
the binary included in releases supports both the KVM and MSHV
hypervisor with runtime detection to identify the correct hypervisor to use.
Snapshot/Restore and Live Migration Improvements
Improvements have been made to the data structures used for both live migration
and snapshot/restore. Unfortunately this has broken compatibility with older
versions (support for migrating between major versions is not yet officially
supported but has worked for some versions.)
Heap Allocation Improvements
Improvements have been made to the volume of heap allocations when running with
virtio-block
devices along with a reduction in the peak heap size.
ch-remote
Improvements
Support for "pinging" the VMM and shutting the VMM down have been added to
ch-remote
.
AArch64
Documentation Integration
The documentation for AArch64
support has been integrated into the main
README.
virtio-block
Counters Enhancement
The counters for the virtio-block
device has extended to include min/mean/max
latency counters.
TCP Offload Control
The virtio-net
device has gained support for controlling the enabling of
checksum and offloading. This allows the device to be used in environments
where the hardware lacks support for the offloading.
Notable Bug Fixes
- Update dependencies including a version of
linux-loader
that addresses an
infinite loop issue (details) - Fix bugs related to
virtio-net
including an integer overflow issue
(#4924, #4949) - Use host
cpuid
information for L2 cache for older KVM on x86 (#4920) - Memory mapped into the guest is now marked as non-dumpable which prevents large core files (#5016)
- Fix QCOW2 refcount table size (#5034)
- Fix unpause support on MSHV in dual binary (#5037)
- Threads inside
virtio
devices are now shutdown on reboot (#5095)
Removals
No functionality has been removed in this release.
Deprecations
- Support for specifying a directory with
MemoryZoneConfig::file
or
MemoryConfig::file
has been deprecated. This was originally used for
supporting huge pages or shared memory backing which is now natively supported
(#5085)
Contributors
Many thanks to everyone who has contributed to our release:
- Bo Chen chen.bo@intel.com
- Claudio Fontana claudio.fontana@gmail.com
- Hao Xu howeyxu@tencent.com
- Henry Wang Henry.Wang@arm.com
- Jinank Jain jinankjain@microsoft.com
- Michael Zhao michael.zhao@arm.com
- Muminul Islam muislam@microsoft.com
- Philipp Schuster philipp.schuster@cyberus-technology.de
- Rob Bradford robert.bradford@intel.com
- Sebastien Boeuf sebastien.boeuf@intel.com
- Shuaiyi Zhang zhangsy28@lenovo.com
- Wei Liu liuwe@microsoft.com
- Yong He alexyonghe@tencent.com
- Yuji Hagiwara yuuzi41@gmail.com
v28.1
This is a bug fix release. The following issues have been addressed:
v28.0
This release has been tracked in our new roadmap project as iteration v28.0.
Community Engagement (Reminder)
Just a reminder that we have a new mailing list to support broader community discussions. Please consider subscribing. We plan to use this to announce a regular meeting for those interested in talking about Cloud
Hypervisor development.
Long Term Support (LTS) Release
This is the first version of Cloud Hypervisor to be released under the LTS release process. Point releases for bug fixes will be made for the next 18 months; live migration and live upgrade will be supported between the point releases of the LTS.
Virtualised TPM Support
Support for adding an emulated CRB TPM has been added. This has it's own TPM documentation.
Transparent Huge Page Support
By default, but controllable through --memory thp=off
if it possible to back the guest memory with Transparent Huge Pages (no file backing/shared=off
) then this will be used resulting in improved boot performance.
README Quick Start Improved
The README has been refreshed with the quick start guide updated to reflect the different firmware options and to recommend the use of pre-built binaries.
Notable Bug Fixes
- Inappropriate Copy-on-Write of pinned pages (e.g. VFIO) leading to higher
memory consumption (#4835) - Multiple
virtio
device bug fixes found through fuzzing (#4859, #4799) - Large QCOW files (> 4TiB) are now supported (#4767)
- Support for > 31 vCPUS on aarch64 (#4863)
- Multiple fixes to OpenAPI specification file (#4720, #4811)
- Programming of the MSI-X table has been optimised leading to faster boot on
newer Linux kernels (#4744) - Error on reboot from race to close TAP devices (#4871)
- Non-spec compliant virtio-block read-only support (#4888)
Removals
The following functionality has been removed:
- Support for non-PVH firmware booting has been removed (#4511)
- I/O ports used for older versions of firmware have been removed (#3926)
- Deprecated API options for kernel/cmdline/initramfs have been removed (#4737)
Contributors
Many thanks to everyone who has contributed to our release:
- Anatol Belski anbelski@linux.microsoft.com
- Bo Chen chen.bo@intel.com
- Fabiano Fidêncio fabiano.fidencio@intel.com
- Jianyong Wu jianyong.wu@arm.com
- Jinank Jain jinankjain@microsoft.com
- Jinrong Liang cloudliang@tencent.com
- lv.mengzhao lv.mengzhao@zte.com.cn
- Michael Zhao michael.zhao@arm.com
- Muminul Islam muislam@microsoft.com
- Praveen K Paladugu prapal@linux.microsoft.com
- Rob Bradford robert.bradford@intel.com
- Sebastien Boeuf sebastien.boeuf@intel.com
- Wei Liu liuwe@microsoft.com
v27.0
This release has been tracked in our new roadmap
project as iteration
v27.0.
Community Engagement
A new mailing list has been created to support broader community discussions.
Please consider subscribing; an
announcement of a regular meeting will be announced via this list shortly.
Prebuilt Packages
Prebuilt packages are now available. Please see this
document
on how to install. These packages also include packages for the different
firmware options available.
Network Device MTU Exposed to Guest
The MTU for the TAP device associated with a virtio-net
device is now exposed
to the guest. If the user provides a MTU with --net mtu=..
then that MTU is
applied to created TAP interfaces. This functionality is also exposed for
vhost-user-net
devices including those created with the reference backend
(#4658, #4676.)
Boot Tracing
Support for generating a trace report for the boot time has been added
including a script for generating an SVG from that trace (#4659.)
Simplified Build Feature Flags
The set of feature flags, for e.g. experimental features, have been simplified:
msvh
andkvm
features provide support for those specific hypervisors
(withkvm
enabled by default),tdx
provides support for Intel TDX; and although there is no MSHV support
now it is now possible to compile with themshv
feature (#4696,)tracing
adds support for boot tracing,guest_debug
now covers both support for gdbing a guest (formerlygdb
feature) and dumping guest memory.
The following feature flags were removed as the functionality was enabled by
default: amx
, fwdebug
, cmos
and common
(#4679, #4632.)
Asynchronous Kernel Loading
AArch64 has gained support for loading the guest kernel asynchronously like
x86-64. (#4538)
GDB Support for AArch64
GDB stub support (accessed through --gdb
under guest_debug
feature) is now
available on AArch64 as well as as x86-64.
Notable Bug Fixes
- This version incorporates a version of
virtio-queue
that addresses an issue
where a rogue guest can potentially DoS the VMM (rust-vmm/vm-virtio#196.) - Improvements around PTY handling for
virtio-console
and serial devices
(#4520, #4533, #4535.) - Improved error handling in virtio devices (#4626, #4605, #4509, #4631, #4697)
Deprecations
Deprecated features will be removed in a subsequent release and users should
plan to use alternatives.
- Booting legacy firmware (compiled without a PVH header) has been deprecated.
All the firmware options (Cloud Hypervisor OVMF and Rust Hypervisor Firmware)
support booting with PVH so support for loading firmware in a legacy mode is no
longer needed. This functionality will be removed in the next release.
Contributors
Many thanks to everyone who has contributed to our release:
- Anatol Belski anbelski@linux.microsoft.com
- Bo Chen chen.bo@intel.com
- James O. D. Hunt james.o.hunt@intel.com
- Jianyong Wu jianyong.wu@arm.com
- Markus Napierkowski markus.napierkowski@cyberus-technology.de
- Michael Zhao michael.zhao@arm.com
- Nuno Das Neves nudasnev@microsoft.com
- Rob Bradford robert.bradford@intel.com
- Sebastien Boeuf sebastien.boeuf@intel.com
- Smit Gardhariya sgardhariya@microsoft.com
- Wei Liu liuwe@microsoft.com
v26.0
This release has been tracked through the v26.0
project.
SMBIOS Improvements via --platform
--platform
and the appropriate API structure has gained support for supplying
OEM strings (primarily used to communicate metadata to systemd in the guest)
(#4319, #4446) and support for specifying the UUID (#4389.)
Unified Binary MSHV and KVM Support
Support for both the MSHV and KVM hypervisors can be compiled into the same
binary with the detection of the hypervisor to use made at runtime.
Notable Bug Fixes
- The prefetchable flag is preserved on BARs for VFIO devices (#4353, #4454)
- PCI Express capabilties for functionality we do not support are now filtered
out (#4456) - GDB breakpoint support is more reliable (#4354, #4363)
SIGINT
andSIGTERM
signals are now handled before the VM has booted
(#4269, #4293)- Multiple API event loop handling bug fixes (#4309, #4362)
- Incorrect assumptions in virtio queue numbering were addressed, allowing
thevirtio-fs
driver in OVMF to be used (#4341, #4314) - VHDX file format header fix (#4291)
- The same VFIO device cannot be added twice (#4453, #4463)
- SMBIOS tables were being incorrectly generated (#4442)
Deprecations
Deprecated features will be removed in a subsequent release and users should
plan to use alternatives.
- The top-level
kernel
andinitramfs
members on theVmConfig
have been
moved inside aPayloadConfig
as thepayload
member. The OpenAPI document
has been updated to reflect the change and the old API members continue to
function and are mapped to the new version. The expectation is that these old
versions will be removed in the v28.0 release.
Removals
The following functionality has been removed:
- The unused
poll_queue
parameter has been removed from--disk
and
equivalent. This was residual from the removal of thevhost-user-block
spawning feature (#4402.)
Contributors
Many thanks to everyone who has contributed to our release:
- Alyssa Ross hi@alyssa.is
- Anatol Belski ab@php.net
- Archana Shinde archana.m.shinde@intel.com
- Bo Chen chen.bo@intel.com
- lizhaoxin1 Lxiaoyouling@163.com
- Maximilian Nitsch maximilian.nitsch@d3tn.com
- Michael Zhao michael.zhao@arm.com
- Rob Bradford robert.bradford@intel.com
- Sebastien Boeuf sebastien.boeuf@intel.com
- Steven Dake sdake@lambdal.com
- Wei Liu liuwe@microsoft.com
v25.0
This release has been tracked through the v25.0
project.
ch-remote
Improvements
The ch-remote
command has gained support for creating the VM from a JSON
config and support for booting and deleting the VM from the VMM.
VM "Coredump" Support
Under the guest_debug
feature flag it is now possible to extract the memory
of the guest for use in debugging with e.g. the crash
utility. (#4012)
Notable Bug Fixes
- Always restore console mode on exit (#4249, #4248)
- Restore vCPUs in numerical order which fixes aarch64 snapshot/restore (#4244)
- Don't try and configure
IFF_RUNNING
on TAP devices (#4279) - Propagate configured queue size through to vhost-user backend (#4286)
- Always Program vCPU CPUID before running the vCPU to fix running on Linux
5.16 (#4156) - Enable ACPI MADT "Online Capable" flag for hotpluggable vCPUs to fix newer
Linux guest
Removals
The following functionality has been removed:
- The
mergeable
option from thevirtio-pmem
support has been removed
(#3968) - The
dax
option from thevirtio-fs
support has been removed (#3889)
Contributors
Many thanks to everyone who has contributed to our release:
- Dylan Bargatze dbargatz@users.noreply.github.com
- Jinank Jain jinankjain@microsoft.com
- Michael Zhao michael.zhao@arm.com
- Rob Bradford robert.bradford@intel.com
- Sebastien Boeuf sebastien.boeuf@intel.com
- Wei Liu liuwe@microsoft.com
- Yi Wang wang.yi59@zte.com.cn
v24.0
This release has been tracked through the v24.0
project.
Bypass Mode for virtio-iommu
virtio-iommu
specification describes how a device can be attached by default
to a bypass domain. This feature is particularly helpful for booting a VM with
guest software which doesn't support virtio-iommu
but still need to access
the device. Now that Cloud Hypervisor supports this feature, it can boot a VM
with Rust Hypervisor Firmware or OVMF even if the virtio-block
device exposing
the disk image is placed behind a virtual IOMMU.
Ensure Identifiers Uniqueness
Multiple checks have been added to the code to prevent devices with identical
identifiers from being created, and therefore avoid unexpected behaviors at boot
or whenever a device was hot plugged into the VM.
Sparse Mmap support
Sparse mmap support has been added to both VFIO and vfio-user devices. This
allows the device regions that are not fully mappable to be partially mapped.
And the more a device region can be mapped into the guest address space, the
fewer VM exits will be generated when this device is accessed. This directly
impacts the performance related to this device.
Expose Platform Serial Number
A new serial_number
option has been added to --platform
, allowing a user to
set a specific serial number for the platform. This number is exposed to the
guest through the SMBIOS.
Notable Bug Fixes
- Fix loading RAW firmware (#4072)
- Reject compressed QCOW images (#4055)
- Reject virtio-mem resize if device is not activated (#4003)
- Fix potential mmap leaks from VFIO/vfio-user MMIO regions (#4069)
- Fix algorithm finding HOB memory resources (#3983)
Notable Improvements
- Refactor interrupt handling (#4083)
- Load kernel asynchronously (#4022)
- Only create ACPI memory manager DSDT when resizable (#4013)
Deprecations
Deprecated features will be removed in a subsequent release and users should
plan to use alternatives
- The
mergeable
option from thevirtio-pmem
support has been deprecated
(#3968) - The
dax
option from thevirtio-fs
support has been deprecated (#3889)
New on the Website
A new blog post Achieving Bare Metal Performance Within a Virtual
Machine
has been added to the Cloud Hypervisor website.
Contributors
Many thanks to everyone who has contributed to our release:
- Anatol Belski anbelski@linux.microsoft.com
- Bo Chen chen.bo@intel.com
- Fabiano Fidêncio fabiano.fidencio@intel.com
- LiHui andrewli@kubesphere.io
- Maksym Pavlenko pavlenko.maksym@gmail.com
- Rob Bradford robert.bradford@intel.com
- Sebastien Boeuf sebastien.boeuf@intel.com
- Steven Dake steven.dake@gmail.com
- Vincent Batts vbatts@hashbangbash.com
- Wei Liu liuwe@microsoft.com
v23.1
This is a bug fix release. The following issues have been addressed:
- Add some missing seccomp rules
- Remove
virtio-fs
filesystem entries from config on removal - Do not delete API socket on API server start (#4026)
- Reject
virtio-mem
resize if the guest doesn't activate the device - Fix OpenAPI naming of I/O throttling knobs
v23.0
This release has been tracked through the v23.0
project.
vDPA Support
A vDPA device has a datapath that complies with the virtio specification but
with a vendor specific control path. The addition of --vdpa
and the REST API
equivalent allows the use of these devices with Cloud Hypervisor.
Updated OS Support list
The list of officially supported and tested OS versions has been updated to
include Ubuntu "jammy" 22.04 and EOLed versions removed.
AArch64
Memory Map Improvements
The memory map when running on AArch64
has been improved for the handling of
the UEFI region which means that the booted guest OS now has full access to its
allocated RAM. (#3938)
AMX
Support
Under a compile time gate of amx
it is possible compile in support for the
AMX
instruction set extension for guest use. This also requires runtime
enabling with --cpu features=amx
.
Notable Bug Fixes
- Generate error when incorrect HTTP method used for some API endpoints (#3887)
- CMOS based reset is now available to support rebooting on "jammy" (#3914)
- ACPI tables are not produced for memory hotplug when running with
virtio-mem
(#3883) virtio-iommu
backed PCI segments are now comprehensively placed behind the
vIOMMU (#3870)- Seccomp rules have been extended for
virtio-fs
to support direct access
(#3848)
Deprecations
Deprecated features will be removed in a subsequent release and users should
plan to use alternatives
- The
mergeable
option from thevirtio-pmem
support has been deprecated
(#3968) - The
dax
option from thevirtio-fs
support has been deprecated (#3889)
Contributors
Many thanks to everyone who has contributed to our release:
- Bo Chen chen.bo@intel.com
- Fabiano Fidêncio fabiano.fidencio@intel.com
- Henry Wang Henry.Wang@arm.com
- Jianyong Wu jianyong.wu@arm.com
- LiHui andrewli@kubesphere.io
- Michael Zhao michael.zhao@arm.com
- Rob Bradford robert.bradford@intel.com
- Sebastien Boeuf sebastien.boeuf@intel.com
- Wei Liu liuwe@microsoft.com
- William Douglas william.douglas@intel.com
- Yi Wang wang.yi59@zte.com.cn