Include UEFI loader and nvram in box

[michaelbeaumont]

Is your feature request related to a problem? Please describe.
I'm building QEMU images on a UEFI machine and booting with EFISTUB, so I need to add a UEFI boot entry and use the vars when I boot the machine.

Describe the solution you'd like
Be able to include UEFI loader and nvram in a box. E.g.

{ "disks": [{"loader": "..."}, {"nvram": "..."}, {"path": "box.img"}] }

Describe alternatives you've considered
I'm not overly familiar with all of UEFI but maybe there's a way to add a boot entry that doesn't depend on the UUID of the disk and thus doesn't ever change when rebuilding the image.
EDIT: We can always set the same GUID when creating the disk and use this GUID in the EFI boot entry. Then we'd only need an initial copy of the efivars once.

[trinitronx]

Perhaps this feature could be an extension to what was provided in f498f10, and 64a0764. Example config here.

If vagrant-libvirt could specify loader and nvram as in the "Auto-selection Example" below, we could avoid the following problem...

Problem: Vagrant .box UEFI firmware dependency

Right now, there are options loader and nvram which by default will create XML nodes using whatever the provided text from a Vagrantfile has. For example:

The following Vagrantfile has options loader and nvram defined:

Vagrant.configure("2") do |config|

  config.vm.provider :libvirt do |libvirt|

# [... SNIP ..]

    libvirt.loader = "/usr/share/OVMF/x64/OVMF_CODE.fd"
    libvirt.nvram = "/usr/share/OVMF/x64/OVMF_VARS.fd"

# [... SNIP ..]

  end
end

The above Vagrantfile options will result in a VM (a.k.a "domain") being defined in libvirt with the following XML:

<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <name>default-vagrant-vm-name_default</name>
  
  <!-- ... SNIP ... -->

  <os>
    <type arch='x86_64' machine='pc-q35-8.1'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/OVMF/x64/OVMF_CODE.fd</loader>
    <nvram>/usr/share/OVMF/x64/OVMF_VARS.fd</nvram>
    <boot dev='hd'/>
    <bootmenu enable='no'/>
  </os>

  <!-- ... SNIP ... -->

</domain>

Packaging Vagrant VMs into a .box file requires one to make a choice for such VMs that require UEFI firmware:

1. Package the OVMF_*.fd firmware files inside the .box directory, and somehow auto-detect the path to the installed box under ~/.vagrant.d/ at runtime.
2. Rely on OS-provided UEFI firmware files (e.g. TianoCore EDK2 files under /usr/share/OVMF/OVMF_VARS.fd, or perhaps /usr/share/OVMF/x64/OVMF_CODE.fd), and hope that those hardcoded paths match the location on the target host system.
3. Some combination of 1 & 2 with auto-detect and/or fallback options

Better Solution: UEFI auto-selection

Modern libvirt & qemu have support for UEFI firmware auto-selection, which automatically copies from master loader & nvram template files provided by the host OS-packaged EDK2 in: /usr/share/OVMF/, and /usr/share/qemu/firmware/*.json.

For example, on an up-to-date ArchLinux system with libvirt, qemu-full, and edk2-ovmf packages installed we have these files:

UEFI firmware from TianoCore's edk2-ovmf package:

tree /usr/share/OVMF/                                                                                                                    

/usr/share/OVMF/
├── aarch64
│   ├── QEMU_CODE.fd
│   ├── QEMU_EFI.fd
│   └── QEMU_VARS.fd
├── arm
│   ├── QEMU_CODE.fd
│   ├── QEMU_EFI.fd
│   └── QEMU_VARS.fd
├── ia32
│   ├── OVMF.4m.fd
│   ├── OVMF_CODE.4m.fd
│   ├── OVMF_CODE.csm.4m.fd
│   ├── OVMF_CODE.csm.fd
│   ├── OVMF_CODE.fd
│   ├── OVMF_CODE.secboot.4m.fd
│   ├── OVMF_CODE.secboot.fd
│   ├── OVMF.fd
│   ├── OVMF_VARS.4m.fd
│   └── OVMF_VARS.fd
└── x64
    ├── MICROVM.4m.fd
    ├── MICROVM.fd
    ├── OVMF.4m.fd
    ├── OVMF_CODE.4m.fd
    ├── OVMF_CODE.csm.4m.fd
    ├── OVMF_CODE.csm.fd
    ├── OVMF_CODE.fd
    ├── OVMF_CODE.secboot.4m.fd
    ├── OVMF_CODE.secboot.fd
    ├── OVMF.fd
    ├── OVMF_VARS.4m.fd
    └── OVMF_VARS.fd

5 directories, 28 files

QEMU .json specification files used for auto-selection:

$ tree /usr/share/qemu/firmware/

/usr/share/qemu/firmware/
├── 50-edk2-ovmf-i386-secure-4m.json
├── 50-edk2-ovmf-i386-secure.json
├── 50-edk2-ovmf-x86_64-secure-4m.json
├── 50-edk2-ovmf-x86_64-secure.json
├── 60-edk2-aarch64.json
├── 60-edk2-arm.json
├── 60-edk2-ovmf-i386-4m.json
├── 60-edk2-ovmf-i386.json
├── 60-edk2-ovmf-microvm-4m.json
├── 60-edk2-ovmf-microvm.json
├── 60-edk2-ovmf-x86_64-4m.json
├── 60-edk2-ovmf-x86_64.json
├── 70-edk2-ovmf-i386-csm-4m.json
├── 70-edk2-ovmf-i386-csm.json
├── 70-edk2-ovmf-x86_64-csm-4m.json
├── 70-edk2-ovmf-x86_64-csm.json
├── 80-edk2-ovmf-ia32-on-x86_64-secure-4m.json
├── 80-edk2-ovmf-ia32-on-x86_64-secure.json
├── 81-edk2-ovmf-ia32-on-x86_64-4m.json
├── 81-edk2-ovmf-ia32-on-x86_64.json
├── 82-edk2-ovmf-ia32-on-x86_64-csm-4m.json
└── 82-edk2-ovmf-ia32-on-x86_64-csm.json

1 directory, 22 files

As an example of what these .json files contain...

Expand to see contents of 60-edk2-ovmf-x86_64.json

{
    "description": "x64 UEFI for x86_64",
    "interface-types": [
        "uefi"
    ],
    "mapping": {
        "device": "flash",
        "executable": {
            "filename": "/usr/share/edk2/x64/OVMF_CODE.fd",
            "format": "raw"
        },
        "nvram-template": {
            "filename": "/usr/share/edk2/x64/OVMF_VARS.fd",
            "format": "raw"
        }
    },
    "targets": [
        {
            "architecture": "x86_64",
            "machines": [
                "pc-i440fx-*",
                "pc-q35-*"
            ]
        }
    ],
    "features": [
        "acpi-s3",
        "acpi-s4",
        "amd-sev",
        "verbose-dynamic"
    ],
    "tags": [

    ]
}

Querying via libvirt's domcapabilities API we can see the following:

$ virsh domcapabilities --machine pc-q35-5.1 | xmllint --xpath '/domainCapabilities/os' -

<os supported="yes">
    <enum name="firmware">
      <value>efi</value>
    </enum>
    <loader supported="yes">
      <value>/usr/share/edk2/x64/OVMF_CODE.secboot.4m.fd</value>
      <value>/usr/share/edk2/x64/OVMF_CODE.secboot.fd</value>
      <value>/usr/share/edk2/x64/OVMF_CODE.4m.fd</value>
      <value>/usr/share/edk2/x64/OVMF_CODE.fd</value>
      <value>/usr/share/edk2/x64/OVMF_CODE.csm.4m.fd</value>
      <value>/usr/share/edk2/x64/OVMF_CODE.csm.fd</value>
      <value>/usr/share/edk2/ia32/OVMF_CODE.secboot.4m.fd</value>
      <value>/usr/share/edk2/ia32/OVMF_CODE.secboot.fd</value>
      <value>/usr/share/edk2/ia32/OVMF_CODE.4m.fd</value>
      <value>/usr/share/edk2/ia32/OVMF_CODE.fd</value>
      <value>/usr/share/edk2/ia32/OVMF_CODE.csm.4m.fd</value>
      <value>/usr/share/edk2/ia32/OVMF_CODE.csm.fd</value>
      <enum name="type">
        <value>rom</value>
        <value>pflash</value>
      </enum>
      <enum name="readonly">
        <value>yes</value>
        <value>no</value>
      </enum>
      <enum name="secure">
        <value>yes</value>
        <value>no</value>
      </enum>
    </loader>
  </os>

Auto-selection Example

The files & domcapabilities output above gives us an idea of what UEFI firmwares are supported by this machine type (pc-q35-5.1), and how libvirt will select one for us automatically if we specify in a VM as follows:

<domain type="kvm">
  <!-- ... -->
  <os firmware="efi">
    <type arch="x86_64" machine="pc-q35-5.1">hvm</type>
    <loader secure="no"/>
    <boot dev="hda"/>
  </os>
  <!-- ... -->
</domain>

 

Right now, vagrant-libvirt has no concept of this auto-selection behavior and no way to request or specify it in the generated libvirt VM XML. Adding this feature would allow for decoupling UEFI firmware from the .box file contents and avoiding the problem of having to package & detect file paths at Vagrantfile runtime.

[trinitronx]

Some things to note about using auto-selection:

1. The numeric ordering of files in /usr/share/qemu/firmware does matter
2. Some UEFI firmwares are still tricky to filter out or select based on the available JSON criteria alone

For example, if we try to pick one with:

  <os firmware='efi'>
    <type arch='x86_64' machine='pc-q35-8.1'>hvm</type>
    <firmware>
      <feature enabled='no' name='enrolled-keys'/>
      <feature enabled='no' name='secure-boot'/>
    </firmware>
    <loader readonly='yes' type='pflash'/>
    <nvram/>
    <bootmenu enable='no'/>
  </os>

This is roughly equivalent to the following jq query, which filters on these same JSON criteria:

cat /usr/share/qemu/firmware/* | jq -s 'map(select( (has("features") and (any(.features[]; . == "secure-boot")|not)) and (has("features") and (any(.features[]; . == "enrolled-keys")|not)) and (has("targets") and any(.targets[]; .architecture == "x86_64" and any(.machines[]; . == "pc-q35-*")) )    ))'

This filtering still results in 8 firmwares that meet the criteria:

Expand for jq query results

[
  {
    "description": "x64 UEFI for x86_64, 4MB FD",
    "interface-types": [
      "uefi"
    ],
    "mapping": {
      "device": "flash",
      "executable": {
        "filename": "/usr/share/edk2/x64/OVMF_CODE.4m.fd",
        "format": "raw"
      },
      "nvram-template": {
        "filename": "/usr/share/edk2/x64/OVMF_VARS.4m.fd",
        "format": "raw"
      }
    },
    "targets": [
      {
        "architecture": "x86_64",
        "machines": [
          "pc-i440fx-*",
          "pc-q35-*"
        ]
      }
    ],
    "features": [
      "acpi-s3",
      "acpi-s4",
      "amd-sev",
      "verbose-dynamic"
    ],
    "tags": []
  },
  {
    "description": "x64 UEFI for x86_64",
    "interface-types": [
      "uefi"
    ],
    "mapping": {
      "device": "flash",
      "executable": {
        "filename": "/usr/share/edk2/x64/OVMF_CODE.fd",
        "format": "raw"
      },
      "nvram-template": {
        "filename": "/usr/share/edk2/x64/OVMF_VARS.fd",
        "format": "raw"
      }
    },
    "targets": [
      {
        "architecture": "x86_64",
        "machines": [
          "pc-i440fx-*",
          "pc-q35-*"
        ]
      }
    ],
    "features": [
      "acpi-s3",
      "acpi-s4",
      "amd-sev",
      "verbose-dynamic"
    ],
    "tags": []
  },
  {
    "description": "x64 UEFI for x86_64, with CSM support, 4MB FD",
    "interface-types": [
      "uefi"
    ],
    "mapping": {
      "device": "flash",
      "executable": {
        "filename": "/usr/share/edk2/x64/OVMF_CODE.csm.4m.fd",
        "format": "raw"
      },
      "nvram-template": {
        "filename": "/usr/share/edk2/x64/OVMF_VARS.4m.fd",
        "format": "raw"
      }
    },
    "targets": [
      {
        "architecture": "x86_64",
        "machines": [
          "pc-i440fx-*",
          "pc-q35-*"
        ]
      }
    ],
    "features": [
      "acpi-s3",
      "acpi-s4",
      "amd-sev",
      "verbose-dynamic"
    ],
    "tags": []
  },
  {
    "description": "x64 UEFI for x86_64, with CSM support",
    "interface-types": [
      "uefi"
    ],
    "mapping": {
      "device": "flash",
      "executable": {
        "filename": "/usr/share/edk2/x64/OVMF_CODE.csm.fd",
        "format": "raw"
      },
      "nvram-template": {
        "filename": "/usr/share/edk2/x64/OVMF_VARS.fd",
        "format": "raw"
      }
    },
    "targets": [
      {
        "architecture": "x86_64",
        "machines": [
          "pc-i440fx-*",
          "pc-q35-*"
        ]
      }
    ],
    "features": [
      "acpi-s3",
      "acpi-s4",
      "amd-sev",
      "verbose-dynamic"
    ],
    "tags": []
  },
  {
    "description": "IA32 UEFI for x86_64, 4MB FD",
    "interface-types": [
      "uefi"
    ],
    "mapping": {
      "device": "flash",
      "executable": {
        "filename": "/usr/share/edk2/ia32/OVMF_CODE.4m.fd",
        "format": "raw"
      },
      "nvram-template": {
        "filename": "/usr/share/edk2/ia32/OVMF_VARS.4m.fd",
        "format": "raw"
      }
    },
    "targets": [
      {
        "architecture": "x86_64",
        "machines": [
          "pc-i440fx-*",
          "pc-q35-*"
        ]
      }
    ],
    "features": [
      "acpi-s3",
      "acpi-s4",
      "verbose-dynamic"
    ],
    "tags": []
  },
  {
    "description": "IA32 UEFI for x86_64",
    "interface-types": [
      "uefi"
    ],
    "mapping": {
      "device": "flash",
      "executable": {
        "filename": "/usr/share/edk2/ia32/OVMF_CODE.fd",
        "format": "raw"
      },
      "nvram-template": {
        "filename": "/usr/share/edk2/ia32/OVMF_VARS.fd",
        "format": "raw"
      }
    },
    "targets": [
      {
        "architecture": "x86_64",
        "machines": [
          "pc-i440fx-*",
          "pc-q35-*"
        ]
      }
    ],
    "features": [
      "acpi-s3",
      "acpi-s4",
      "verbose-dynamic"
    ],
    "tags": []
  },
  {
    "description": "IA32 UEFI for x86_64, with CSM, 4MB FD",
    "interface-types": [
      "uefi"
    ],
    "mapping": {
      "device": "flash",
      "executable": {
        "filename": "/usr/share/edk2/ia32/OVMF_CODE.csm.4m.fd",
        "format": "raw"
      },
      "nvram-template": {
        "filename": "/usr/share/edk2/ia32/OVMF_VARS.4m.fd",
        "format": "raw"
      }
    },
    "targets": [
      {
        "architecture": "x86_64",
        "machines": [
          "pc-i440fx-*",
          "pc-q35-*"
        ]
      }
    ],
    "features": [
      "acpi-s3",
      "acpi-s4",
      "verbose-dynamic"
    ],
    "tags": []
  },
  {
    "description": "IA32 UEFI for x86_64, with CSM",
    "interface-types": [
      "uefi"
    ],
    "mapping": {
      "device": "flash",
      "executable": {
        "filename": "/usr/share/edk2/ia32/OVMF_CODE.csm.fd",
        "format": "raw"
      },
      "nvram-template": {
        "filename": "/usr/share/edk2/ia32/OVMF_VARS.fd",
        "format": "raw"
      }
    },
    "targets": [
      {
        "architecture": "x86_64",
        "machines": [
          "pc-i440fx-*",
          "pc-q35-*"
        ]
      }
    ],
    "features": [
      "acpi-s3",
      "acpi-s4",
      "verbose-dynamic"
    ],
    "tags": []
  }
]

The first one in order is "x64 UEFI for x86_64, 4MB FD", with loader: /usr/share/edk2/x64/OVMF_CODE.4m.fd and nvram: /usr/share/edk2/x64/OVMF_VARS.4m.fd. The resulting libvirt-generated VM ("domain") XML for a VM with name macOS-monterey-base-clone would therefore be:

  <os firmware='efi'>
    <type arch='x86_64' machine='pc-q35-8.1'>hvm</type>
    <firmware>
      <feature enabled='no' name='enrolled-keys'/>
      <feature enabled='no' name='secure-boot'/>
    </firmware>
    <loader readonly='yes' type='pflash'>/usr/share/edk2/x64/OVMF_CODE.4m.fd</loader>
    <nvram template='/usr/share/edk2/x64/OVMF_VARS.4m.fd'>/var/lib/libvirt/qemu/nvram/macOS-monterey-base-clone_VARS.fd</nvram>
    <bootmenu enable='no'/>
  </os>

So, using UEFI firmware auto-selection in practice may be a bit harder to precisely select for a specific one, given the fact that the tags and features attribute arrays are still very similar across all of these remaining 8 firmware images.

That being said... there is still probably a use case for both firmware auto-selection, and packaging UEFI loader & nvram images in a Vagrant .box. Auto-selection currently isn't supported with vagrant-libvirt today, but one could hack together a base Vagrantfile inside the .box which specifies which loader and nvram path to use. However, currently the nvram one is problematic because it will overwrite any path given since the VM will treat this file as a read-write pflash. This is because vagrant-libvirt doesn't yet support setting an nvram template (e.g. <nvram template="thing here">...</nvram>)

For this to work, vagrant-libvirt would have to support generating libvirt XML with the template attribute on nvram. For example:

    <loader readonly='yes' type='pflash'>/usr/share/edk2/x64/OVMF_CODE.fd</loader>
    <nvram template='/usr/share/edk2/x64/OVMF_VARS.fd'>/var/lib/libvirt/qemu/nvram/macOS-monterey-base-clone_VARS.fd</nvram>

The nvram template and loader paths could be detected inside a .box-packaged base Vagrantfile:

# Once Vagrant unpacks the .box including this Vagrantfile,
# This path would be something like: ~/.vagrant.d/boxes/username-VAGRANTSLASH-box-name 
BOX_DIR = File.expand_path(File.dirname(__FILE__))

Vagrant.configure("2") do |config|

  config.vm.provider :libvirt do |libvirt|

  # [...SNIP...]

    # Assuming these UEFI firmware images are found under ~/.vagrant.d/boxes/username-VAGRANTSLASH-box-name/OVMF_{CODE,VARS}.fd
    libvirt.loader = "#{BOX_DIR}/OVMF_CODE.fd"
    libvirt.nvram = { template: "#{BOX_DIR}/OVMF_VARS.fd" }  ## This syntax does not currently work... but just an idea of some kind of future way to specify `template` for nvram

  # [...SNIP...]

  end
end

If nvram template was supported, this would result in the VM being created using the pre-packaged OVMF_VARS.fd and OVMF_CODE.fd files. These would be placed inside the .box (a BSD-tar-compatible .tar.gz tarball). Using template is needed so each VM based on this same Vagrant .box won't conflict with all others by writing to OVMF_VARS.fd nvram file. Currently, the way vagrant-libvirt generates the nvram XML results in a read-write pflash, which will overwrite any OVMF_VARS.fd file passed to it:

<nvram>/usr/share/OVMF/x64/OVMF_VARS.fd</nvram>  <!-- This will overwrite the OS-provided OVMF_VARS.fd!  Better to pass in a copy: /var/lib/libvirt/qemu/nvram/vm_locally_owned_copy_of_OVMF_VARS.fd --> 

[Ckarles]

This would really help with the deliverability of libvirt boxes. Anything we can do to motivate a design decision?

On a related topic, what is the best way to document this loader/nvram external requirement? For example, is there a way to flag these 2 parameters in the box' Vagrantfile in order to display an error message indicating these 2 parameters are required when triggering to use the box?