openvmm, virt_kvm: add support for snp with linux direct boot#3970
openvmm, virt_kvm: add support for snp with linux direct boot#3970chris-oo wants to merge 16 commits into
Conversation
|
This PR modifies files containing For more on why we check whole files, instead of just diffs, check out the Rustonomicon |
There was a problem hiding this comment.
Pull request overview
Adds initial AMD SEV-SNP support for Linux direct boot on the virt_kvm backend, including guestmemfd-backed private memory, SNP launch measurement/import of initial pages, and CLI/config validation to keep the supported configuration surface small (virtio-over-PCIe only).
Changes:
- Introduces a new
BaseChipsetType::EnlightenedLinuxDirectpath intended for SNP Linux direct boot. - Implements KVM SNP plumbing:
/dev/sevinit, initial page acceptance/launch update flow, and handling ofKVM_HC_MAP_GPA_RANGE. - Adds CLI/config validation and a loader-side “pre-accept all RAM” bring-up hack for SNP direct boot.
Reviewed changes
Copilot reviewed 12 out of 13 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| vmm_core/vm_manifest_builder/src/lib.rs | Adds new chipset type and build path for “enlightened” Linux direct boot; adds missing PCI config port stubs. |
| vmm_core/virt_kvm/src/snp.rs | Implements SNP launch flow (launch start/update/finish) and page import handling. |
| vmm_core/virt_kvm/src/memory.rs | Adds KVM hypercall-backed memory attribute switching and private-range resolution helpers. |
| vmm_core/virt_kvm/src/lib.rs | Extends KvmError and KvmPartitionInner state to support SNP launch and SEV device tracking. |
| vmm_core/virt_kvm/src/arch/x86_64/snp.rs | Adds mapping from InitialPageImportType to SNP launch page types (with unit tests). |
| vmm_core/virt_kvm/src/arch/x86_64/mod.rs | Wires SNP isolation into KVM VM creation, guestmemfd backing, hypercall handling, and SEV termination reporting. |
| vmm_core/virt_kvm/Cargo.toml | Adds libc dependency and test_with_tracing dev-dependency. |
| openvmm/openvmm_entry/src/lib.rs | Selects the new chipset type for --isolation snp and validates SNP configs post-build. |
| openvmm/openvmm_entry/src/cli_args.rs | Exposes --isolation snp and adds CLI-side validation + tests for SNP option constraints. |
| openvmm/openvmm_core/src/worker/vm_loaders/linux.rs | Adds SNP-only logic to import/accept remaining RAM pages for direct boot (with unit test). |
| openvmm/openvmm_core/src/worker/dispatch.rs | Adds runtime validation gates for SNP guestmemfd limitations. |
| openvmm/openvmm_core/Cargo.toml | Adds test_with_tracing dev-dependency for new tests. |
| Cargo.lock | Locks new dependency additions (libc, test_with_tracing) where applicable. |
Create SNP KVM VMs with guestmemfd-backed memory, wire SEV-SNP launch start/update/finish, handle KVM_HC_MAP_GPA_RANGE conversions, and expose --isolation snp for direct Linux boot. Keep the all-RAM launch workaround documented as bring-up-only until boot modes can accept memory themselves. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
735bc97 to
affa0bd
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 13 out of 14 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (1)
vmm_core/virt_kvm/src/snp.rs:108
sev_snp_launch_updateis invoked withgpa / HV_PAGE_SIZEandlen = page.range.len()without validating that the range is 4K-aligned and a multiple of 4K. If a caller provides an unaligned range, we’ll compute the wrong PFN and/or pass an invalid length to KVM (and the existingUnalignedSnpLaunchRangeerror is never used).
let gpa = page.range.start();
tracing::trace!(
gpa,
len = page.range.len(),
?kvm_page_type,
| let range = MemoryRange::new(gpa..end); | ||
| if !self.ram_ranges.iter().any(|ram| ram.contains(&range)) { | ||
| return Err(KvmError::InvalidMapGpaRange); | ||
| } |
Expose the effective SNP C-bit through x86 partition capabilities and pass it into the Linux loader. Emit the confidential bit directly in identity-map links and leaves, removing KVM's loader-tag-based post-import mutation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 93dd8dd4-0267-413f-91a0-6ead70f97222
affa0bd to
e7f9468
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 17 out of 18 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (1)
vmm_core/virt_kvm/src/snp.rs:73
- SNP launch update assumes page GPA/length are 4KiB-aligned (e.g.,
gfn_startis computed viagpa / HV_PAGE_SIZE). Today there’s no explicit validation, and theUnalignedSnpLaunchRangeerror variant is never used. Add an early alignment/empty-range check so misaligned imports fail deterministically with a clear error instead of passing a truncated GFN into KVM.
let memory = self.memory.lock();
for page in pages {
let kvm_page_type = crate::arch::snp::snp_launch_page_type(page.import_type)?;
let private_range = private_memory_range_from_slots(page.range, &memory.ranges)
| if matches!(self.isolation, Some(IsolationCli::Snp)) { | ||
| if self.uefi { | ||
| anyhow::bail!("SNP isolation currently only supports Linux direct boot"); | ||
| } |
| if cfg.hypervisor.with_hv { | ||
| anyhow::bail!("SNP isolation currently does not support Hyper-V enlightenments"); | ||
| } | ||
| if cfg.hypervisor.with_vtl2.is_some() { | ||
| anyhow::bail!("SNP isolation currently does not support VTL2"); |
| self | ||
| } | ||
|
|
||
| fn attach_missing_pci_config_ports(&mut self, arch: MachineArch) -> &mut Self { |
There was a problem hiding this comment.
What are these ports normally for, the pci bus? Why do we need them?
There was a problem hiding this comment.
linux probes these ports if you have pci (very questionable in a cvm, yes), but if you don't have them you get a ton of missing device spam. i'll add a comment
| .then_some(1 << (physical_address_size - 1)) | ||
| }); | ||
|
|
||
| if cfg.hypervisor.with_isolation == Some(openvmm_defs::config::IsolationType::Snp) { |
There was a problem hiding this comment.
We're gonna need a more scalable solution to different isolations and different support statuses of different features.
There was a problem hiding this comment.
yeah, i'm not sure what the best answer is tbh. hyper-v has a similar issue (and similar mess, imo).
There was a problem hiding this comment.
Probably something more type-driven by the IsolationType enum? Like you'd call config.isolation_type.make_foo_device() and it'd either work or error?
Add a chipset manifest mode for enlightened direct Linux boot that keeps the chipset empty while still wiring serial and missing architectural ports. This provides a reusable manifest shape for paravisor-backed direct boot without adding emulated legacy devices. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> local: keep manifest builder direct-boot shape
Compute the x86 architecture predicate once before selecting the base chipset instead of repeating it in each Linux and Gen2 branch. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 93dd8dd4-0267-413f-91a0-6ead70f97222
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e283add0-66b7-4902-b234-5bb3a7d8ca26
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e283add0-66b7-4902-b234-5bb3a7d8ca26
The KVM-supported CPUID list can contain more entries than the SNP firmware page allows, including leaves whose output is entirely zero. Sanitize entries before serialization and omit all-zero results, which Linux handles as sparse CPUID leaves, while leaving the KVM vCPU CPUID model unchanged. Enforce the firmware's 64-entry limit after compaction and cover post-sanitization filtering and limit handling with unit tests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 93dd8dd4-0267-413f-91a0-6ead70f97222
Validate the SNP and UEFI command-line combination before VM resource and manifest construction so unsupported SNP UEFI boot returns a configuration error instead of reaching the UEFI chipset assertion. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 93dd8dd4-0267-413f-91a0-6ead70f97222
Reject explicit hugetlb backing from both global and per-NUMA memory configuration when SNP isolation is selected. SNP page-state changes operate at 4 KiB granularity, which is incompatible with discarding stale shared hugetlb mappings. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 93dd8dd4-0267-413f-91a0-6ead70f97222
The SNP-only stack constructs guestmemfd backing only for x86_64 guests. Compile the guestmemfd-specific backing types, helpers, and tests only for that guest architecture so aarch64 build and clippy jobs do not fail on dead code. The later CCA stack will enable these paths for aarch64. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 93dd8dd4-0267-413f-91a0-6ead70f97222
Document the SNP isolation CLI mode and its current bring-up constraints, describe the enlightened Linux direct chipset accurately, and replace the broad SNP module dead-code allowance with a targeted expectation for the reserved zero-page variant. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 93dd8dd4-0267-413f-91a0-6ead70f97222
Map loader import types directly to KVM SNP page types and stop inferring hardware ZERO pages from memory contents. IGVM page data without a file payload represents a measured normal private page containing zeroes, and native hardware-zero pages require an explicit format and import type. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 93dd8dd4-0267-413f-91a0-6ead70f97222
Name KVM_HC_MAP_GPA_RANGE arguments before handling them and document that page counts use fixed 4-KiB units while page-size bits describe preferred granularity. Explain why conversions discard the backing no longer selected by KVM. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 93dd8dd4-0267-413f-91a0-6ead70f97222
e7f9468 to
709966c
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 17 out of 18 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (1)
openvmm/openvmm_core/src/worker/dispatch.rs:1208
- The SNP validation block rejects legacy disk paths, but it does not reject unsupported PCI/PCIe devices. That allows non-virtio PCIe devices (or vpci/pci_chipset devices) to slip through in non-CLI configurations, even though SNP bring-up is intended to support only virtio devices attached via PCIe.
if !cfg.floppy_disks.is_empty()
|| !cfg.ide_disks.is_empty()
|| !cfg.virtio_devices.is_empty()
{
anyhow::bail!("KVM SNP guest_memfd does not support disks");
}
| let [eax, ebx, _, _] = cpuid(CpuidFunction::ExtendedSevFeatures.0, 0); | ||
| ExtendedSevFeaturesEax::from(eax) | ||
| .sev_snp() | ||
| .then(|| ExtendedSevFeaturesEbx::from(ebx).cbit_position()) | ||
| } |
| let set_entry = |entry: &mut PageTableEntry, entry_type| { | ||
| entry.set_entry(entry_type); | ||
| if let Some(bit_position) = params.confidential_bit { | ||
| entry.entry |= 1u64 << bit_position; | ||
| } | ||
| }; |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 17 out of 18 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (1)
openvmm/openvmm_core/src/worker/dispatch.rs:1207
- The SNP validation here rejects floppy/IDE/virtio “disks”, but it doesn’t enforce the other SNP bring-up constraints already assumed elsewhere (e.g. only virtio devices attached through PCIe, no vpci/pci-chipset devices). That means a non-CLI caller can construct an SNP config that passes this check but later fails in less obvious ways or violates the intended support matrix.
Consider extending this block to reject non-virtio PCIe devices and other PCI/vPCI devices, matching the CLI validation (and PR description) so the invariant is enforced centrally.
if !cfg.floppy_disks.is_empty()
|| !cfg.ide_disks.is_empty()
|| !cfg.virtio_devices.is_empty()
{
anyhow::bail!("KVM SNP guest_memfd does not support disks");
Allow MAP_GPA_RANGE and CCA RIPAS conversions to span adjacent guestmemfd-backed memory slots. Validate exact slot coverage before changing KVM attributes and discard stale backing for each overlapping segment. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 93dd8dd4-0267-413f-91a0-6ead70f97222
Document SNP launch ownership and lifecycle, CPUID page construction, launch-page translation, and the guestmemfd memory abstractions shared by confidential guests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 93dd8dd4-0267-413f-91a0-6ead70f97222
Drop the CPUID page summary and firmware-diff tracing, including the failure-path page snapshot. KVM launch errors already identify rejected updates without retaining a dedicated debug parser. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 93dd8dd4-0267-413f-91a0-6ead70f97222
80a9e32 to
5c2b9ca
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 17 out of 18 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (1)
vmm_core/virt/src/x86/mod.rs:525
snp_c_bit_from_cpuidreturns the raw CPUID C-bit position without sanity-checking it. If the host (or a buggy VMM) reports a value that falls in the flag bits (e.g. < 12) or aliases NX (bit 63), the loader will OR that bit into every PTE and can generate invalid page tables or unexpected execution permissions.
Consider rejecting obviously-invalid C-bit positions up front (treat as “no SNP C-bit support”).
let [eax, ebx, _, _] = cpuid(CpuidFunction::ExtendedSevFeatures.0, 0);
ExtendedSevFeaturesEax::from(eax)
.sev_snp()
.then(|| ExtendedSevFeaturesEbx::from(ebx).cbit_position())
}
| bsp_cpuid: kvm_cpuid_entries( | ||
| &cpuid, | ||
| &self.config.processor_topology.vp_arch(VpIndex::BSP), | ||
| self.config.processor_topology, | ||
| ), |
| let mut chipset = VmManifestBuilder::new( | ||
| if opt.igvm.is_some() { | ||
| if matches!(opt.isolation, Some(cli_args::IsolationCli::Snp)) { | ||
| BaseChipsetType::EnlightenedLinuxDirect | ||
| } else if opt.igvm.is_some() { | ||
| BaseChipsetType::HclHost |
Add support for direct booting a enlightened Linux kernel on virt_kvm for SNP. This supports only virtio devices exposed over pcie, no hyper-v enlightenments or other devices.
Note that the kernel expects all RAM to be accpeted at kernel start. Since we directly launch the kernel without a boot shim, there is a hack to accept all memory on behalf of the guest, which is very slow. This could be fixed in the future by having a small boot shim to accept memory on behalf of the kernel.