Skip to content

openvmm, virt_kvm: add support for snp with linux direct boot#3970

Open
chris-oo wants to merge 16 commits into
microsoft:mainfrom
chris-oo:snp-cca-pr07b-kvm-snp-launch
Open

openvmm, virt_kvm: add support for snp with linux direct boot#3970
chris-oo wants to merge 16 commits into
microsoft:mainfrom
chris-oo:snp-cca-pr07b-kvm-snp-launch

Conversation

@chris-oo

@chris-oo chris-oo commented Jul 17, 2026

Copy link
Copy Markdown
Member

Add support for direct booting a enlightened Linux kernel on virt_kvm for SNP. This supports only virtio devices exposed over pcie, no hyper-v enlightenments or other devices.

Note that the kernel expects all RAM to be accpeted at kernel start. Since we directly launch the kernel without a boot shim, there is a hack to accept all memory on behalf of the guest, which is very slow. This could be fixed in the future by having a small boot shim to accept memory on behalf of the kernel.

@chris-oo
chris-oo requested a review from a team as a code owner July 17, 2026 23:22
Copilot AI review requested due to automatic review settings July 17, 2026 23:22
@github-actions github-actions Bot added the unsafe Related to unsafe code label Jul 17, 2026
@github-actions

Copy link
Copy Markdown

⚠️ Unsafe Code Detected

This PR modifies files containing unsafe Rust code. Extra scrutiny is required during review.

For more on why we check whole files, instead of just diffs, check out the Rustonomicon

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds initial AMD SEV-SNP support for Linux direct boot on the virt_kvm backend, including guestmemfd-backed private memory, SNP launch measurement/import of initial pages, and CLI/config validation to keep the supported configuration surface small (virtio-over-PCIe only).

Changes:

  • Introduces a new BaseChipsetType::EnlightenedLinuxDirect path intended for SNP Linux direct boot.
  • Implements KVM SNP plumbing: /dev/sev init, initial page acceptance/launch update flow, and handling of KVM_HC_MAP_GPA_RANGE.
  • Adds CLI/config validation and a loader-side “pre-accept all RAM” bring-up hack for SNP direct boot.

Reviewed changes

Copilot reviewed 12 out of 13 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
vmm_core/vm_manifest_builder/src/lib.rs Adds new chipset type and build path for “enlightened” Linux direct boot; adds missing PCI config port stubs.
vmm_core/virt_kvm/src/snp.rs Implements SNP launch flow (launch start/update/finish) and page import handling.
vmm_core/virt_kvm/src/memory.rs Adds KVM hypercall-backed memory attribute switching and private-range resolution helpers.
vmm_core/virt_kvm/src/lib.rs Extends KvmError and KvmPartitionInner state to support SNP launch and SEV device tracking.
vmm_core/virt_kvm/src/arch/x86_64/snp.rs Adds mapping from InitialPageImportType to SNP launch page types (with unit tests).
vmm_core/virt_kvm/src/arch/x86_64/mod.rs Wires SNP isolation into KVM VM creation, guestmemfd backing, hypercall handling, and SEV termination reporting.
vmm_core/virt_kvm/Cargo.toml Adds libc dependency and test_with_tracing dev-dependency.
openvmm/openvmm_entry/src/lib.rs Selects the new chipset type for --isolation snp and validates SNP configs post-build.
openvmm/openvmm_entry/src/cli_args.rs Exposes --isolation snp and adds CLI-side validation + tests for SNP option constraints.
openvmm/openvmm_core/src/worker/vm_loaders/linux.rs Adds SNP-only logic to import/accept remaining RAM pages for direct boot (with unit test).
openvmm/openvmm_core/src/worker/dispatch.rs Adds runtime validation gates for SNP guestmemfd limitations.
openvmm/openvmm_core/Cargo.toml Adds test_with_tracing dev-dependency for new tests.
Cargo.lock Locks new dependency additions (libc, test_with_tracing) where applicable.

Comment thread vmm_core/virt_kvm/src/memory.rs
Comment thread vmm_core/virt_kvm/src/snp.rs
Comment thread vmm_core/virt_kvm/src/arch/x86_64/snp.rs Outdated
Comment thread openvmm/openvmm_entry/src/cli_args.rs
Comment thread openvmm/openvmm_core/src/worker/dispatch.rs
Comment thread vmm_core/vm_manifest_builder/src/lib.rs Outdated
Copilot AI review requested due to automatic review settings July 20, 2026 22:12
@github-actions github-actions Bot added the Guide label Jul 20, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 14 changed files in this pull request and generated 2 comments.

Comment thread vmm_core/virt_kvm/src/snp.rs
Comment thread vmm_core/virt_kvm/src/memory.rs
Create SNP KVM VMs with guestmemfd-backed memory, wire SEV-SNP launch start/update/finish, handle KVM_HC_MAP_GPA_RANGE conversions, and expose --isolation snp for direct Linux boot. Keep the all-RAM launch workaround documented as bring-up-only until boot modes can accept memory themselves.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 20, 2026 22:29
@chris-oo
chris-oo force-pushed the snp-cca-pr07b-kvm-snp-launch branch from 735bc97 to affa0bd Compare July 20, 2026 22:29

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 14 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

vmm_core/virt_kvm/src/snp.rs:108

  • sev_snp_launch_update is invoked with gpa / HV_PAGE_SIZE and len = page.range.len() without validating that the range is 4K-aligned and a multiple of 4K. If a caller provides an unaligned range, we’ll compute the wrong PFN and/or pass an invalid length to KVM (and the existing UnalignedSnpLaunchRange error is never used).
            let gpa = page.range.start();
            tracing::trace!(
                gpa,
                len = page.range.len(),
                ?kvm_page_type,

Comment thread vmm_core/virt_kvm/src/memory.rs Outdated
Comment on lines +306 to +309
let range = MemoryRange::new(gpa..end);
if !self.ram_ranges.iter().any(|ram| ram.contains(&range)) {
return Err(KvmError::InvalidMapGpaRange);
}
Expose the effective SNP C-bit through x86 partition capabilities and pass it into the Linux loader. Emit the confidential bit directly in identity-map links and leaves, removing KVM's loader-tag-based post-import mutation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 93dd8dd4-0267-413f-91a0-6ead70f97222
Copilot AI review requested due to automatic review settings July 21, 2026 17:00
@chris-oo
chris-oo force-pushed the snp-cca-pr07b-kvm-snp-launch branch from affa0bd to e7f9468 Compare July 21, 2026 17:00

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 17 out of 18 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

vmm_core/virt_kvm/src/snp.rs:73

  • SNP launch update assumes page GPA/length are 4KiB-aligned (e.g., gfn_start is computed via gpa / HV_PAGE_SIZE). Today there’s no explicit validation, and the UnalignedSnpLaunchRange error variant is never used. Add an early alignment/empty-range check so misaligned imports fail deterministically with a clear error instead of passing a truncated GFN into KVM.
        let memory = self.memory.lock();
        for page in pages {
            let kvm_page_type = crate::arch::snp::snp_launch_page_type(page.import_type)?;

            let private_range = private_memory_range_from_slots(page.range, &memory.ranges)

Comment on lines +1305 to +1308
if matches!(self.isolation, Some(IsolationCli::Snp)) {
if self.uefi {
anyhow::bail!("SNP isolation currently only supports Linux direct boot");
}
Comment on lines +2057 to +2061
if cfg.hypervisor.with_hv {
anyhow::bail!("SNP isolation currently does not support Hyper-V enlightenments");
}
if cfg.hypervisor.with_vtl2.is_some() {
anyhow::bail!("SNP isolation currently does not support VTL2");
self
}

fn attach_missing_pci_config_ports(&mut self, arch: MachineArch) -> &mut Self {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are these ports normally for, the pci bus? Why do we need them?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

linux probes these ports if you have pci (very questionable in a cvm, yes), but if you don't have them you get a ton of missing device spam. i'll add a comment

Comment thread vmm_core/vm_manifest_builder/src/lib.rs Outdated
Comment thread vmm_core/virt/src/x86/mod.rs
.then_some(1 << (physical_address_size - 1))
});

if cfg.hypervisor.with_isolation == Some(openvmm_defs::config::IsolationType::Snp) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're gonna need a more scalable solution to different isolations and different support statuses of different features.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, i'm not sure what the best answer is tbh. hyper-v has a similar issue (and similar mess, imo).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably something more type-driven by the IsolationType enum? Like you'd call config.isolation_type.make_foo_device() and it'd either work or error?

@github-actions

Copy link
Copy Markdown

chris-oo and others added 5 commits July 21, 2026 14:58
Add a chipset manifest mode for enlightened direct Linux boot that keeps the chipset empty while still wiring serial and missing architectural ports. This provides a reusable manifest shape for paravisor-backed direct boot without adding emulated legacy devices.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

local: keep manifest builder direct-boot shape
Compute the x86 architecture predicate once before selecting the base chipset instead of repeating it in each Linux and Gen2 branch.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 93dd8dd4-0267-413f-91a0-6ead70f97222
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e283add0-66b7-4902-b234-5bb3a7d8ca26
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e283add0-66b7-4902-b234-5bb3a7d8ca26
The KVM-supported CPUID list can contain more entries than the SNP firmware page allows, including leaves whose output is entirely zero. Sanitize entries before serialization and omit all-zero results, which Linux handles as sparse CPUID leaves, while leaving the KVM vCPU CPUID model unchanged.

Enforce the firmware's 64-entry limit after compaction and cover post-sanitization filtering and limit handling with unit tests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 93dd8dd4-0267-413f-91a0-6ead70f97222
chris-oo and others added 6 commits July 21, 2026 14:59
Validate the SNP and UEFI command-line combination before VM resource and manifest construction so unsupported SNP UEFI boot returns a configuration error instead of reaching the UEFI chipset assertion.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 93dd8dd4-0267-413f-91a0-6ead70f97222
Reject explicit hugetlb backing from both global and per-NUMA memory configuration when SNP isolation is selected. SNP page-state changes operate at 4 KiB granularity, which is incompatible with discarding stale shared hugetlb mappings.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 93dd8dd4-0267-413f-91a0-6ead70f97222
The SNP-only stack constructs guestmemfd backing only for x86_64 guests. Compile the guestmemfd-specific backing types, helpers, and tests only for that guest architecture so aarch64 build and clippy jobs do not fail on dead code. The later CCA stack will enable these paths for aarch64.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 93dd8dd4-0267-413f-91a0-6ead70f97222
Document the SNP isolation CLI mode and its current bring-up constraints, describe the enlightened Linux direct chipset accurately, and replace the broad SNP module dead-code allowance with a targeted expectation for the reserved zero-page variant.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 93dd8dd4-0267-413f-91a0-6ead70f97222
Map loader import types directly to KVM SNP page types and stop inferring hardware ZERO pages from memory contents. IGVM page data without a file payload represents a measured normal private page containing zeroes, and native hardware-zero pages require an explicit format and import type.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 93dd8dd4-0267-413f-91a0-6ead70f97222
Name KVM_HC_MAP_GPA_RANGE arguments before handling them and document that page counts use fixed 4-KiB units while page-size bits describe preferred granularity. Explain why conversions discard the backing no longer selected by KVM.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 93dd8dd4-0267-413f-91a0-6ead70f97222
Copilot AI review requested due to automatic review settings July 21, 2026 22:32
@chris-oo
chris-oo force-pushed the snp-cca-pr07b-kvm-snp-launch branch from e7f9468 to 709966c Compare July 21, 2026 22:32

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 17 out of 18 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

openvmm/openvmm_core/src/worker/dispatch.rs:1208

  • The SNP validation block rejects legacy disk paths, but it does not reject unsupported PCI/PCIe devices. That allows non-virtio PCIe devices (or vpci/pci_chipset devices) to slip through in non-CLI configurations, even though SNP bring-up is intended to support only virtio devices attached via PCIe.
            if !cfg.floppy_disks.is_empty()
                || !cfg.ide_disks.is_empty()
                || !cfg.virtio_devices.is_empty()
            {
                anyhow::bail!("KVM SNP guest_memfd does not support disks");
            }

Comment on lines +521 to +525
let [eax, ebx, _, _] = cpuid(CpuidFunction::ExtendedSevFeatures.0, 0);
ExtendedSevFeaturesEax::from(eax)
.sev_snp()
.then(|| ExtendedSevFeaturesEbx::from(ebx).cbit_position())
}
Comment on lines +654 to +659
let set_entry = |entry: &mut PageTableEntry, entry_type| {
entry.set_entry(entry_type);
if let Some(bit_position) = params.confidential_bit {
entry.entry |= 1u64 << bit_position;
}
};
Copilot AI review requested due to automatic review settings July 21, 2026 22:47

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 17 out of 18 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

openvmm/openvmm_core/src/worker/dispatch.rs:1207

  • The SNP validation here rejects floppy/IDE/virtio “disks”, but it doesn’t enforce the other SNP bring-up constraints already assumed elsewhere (e.g. only virtio devices attached through PCIe, no vpci/pci-chipset devices). That means a non-CLI caller can construct an SNP config that passes this check but later fails in less obvious ways or violates the intended support matrix.

Consider extending this block to reject non-virtio PCIe devices and other PCI/vPCI devices, matching the CLI validation (and PR description) so the invariant is enforced centrally.

            if !cfg.floppy_disks.is_empty()
                || !cfg.ide_disks.is_empty()
                || !cfg.virtio_devices.is_empty()
            {
                anyhow::bail!("KVM SNP guest_memfd does not support disks");

chris-oo and others added 3 commits July 21, 2026 16:48
Allow MAP_GPA_RANGE and CCA RIPAS conversions to span adjacent guestmemfd-backed memory slots. Validate exact slot coverage before changing KVM attributes and discard stale backing for each overlapping segment.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 93dd8dd4-0267-413f-91a0-6ead70f97222
Document SNP launch ownership and lifecycle, CPUID page construction, launch-page translation, and the guestmemfd memory abstractions shared by confidential guests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 93dd8dd4-0267-413f-91a0-6ead70f97222
Drop the CPUID page summary and firmware-diff tracing, including the failure-path page snapshot. KVM launch errors already identify rejected updates without retaining a dedicated debug parser.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 93dd8dd4-0267-413f-91a0-6ead70f97222
Copilot AI review requested due to automatic review settings July 21, 2026 23:53
@chris-oo
chris-oo force-pushed the snp-cca-pr07b-kvm-snp-launch branch from 80a9e32 to 5c2b9ca Compare July 21, 2026 23:53
@chris-oo chris-oo changed the title [wip] openvmm, virt_kvm: add support for snp with linux direct boot openvmm, virt_kvm: add support for snp with linux direct boot Jul 21, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 17 out of 18 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

vmm_core/virt/src/x86/mod.rs:525

  • snp_c_bit_from_cpuid returns the raw CPUID C-bit position without sanity-checking it. If the host (or a buggy VMM) reports a value that falls in the flag bits (e.g. < 12) or aliases NX (bit 63), the loader will OR that bit into every PTE and can generate invalid page tables or unexpected execution permissions.

Consider rejecting obviously-invalid C-bit positions up front (treat as “no SNP C-bit support”).

    let [eax, ebx, _, _] = cpuid(CpuidFunction::ExtendedSevFeatures.0, 0);
    ExtendedSevFeaturesEax::from(eax)
        .sev_snp()
        .then(|| ExtendedSevFeaturesEbx::from(ebx).cbit_position())
}

Comment on lines +534 to +538
bsp_cpuid: kvm_cpuid_entries(
&cpuid,
&self.config.processor_topology.vp_arch(VpIndex::BSP),
self.config.processor_topology,
),
Comment on lines 1143 to 1147
let mut chipset = VmManifestBuilder::new(
if opt.igvm.is_some() {
if matches!(opt.isolation, Some(cli_args::IsolationCli::Snp)) {
BaseChipsetType::EnlightenedLinuxDirect
} else if opt.igvm.is_some() {
BaseChipsetType::HclHost
@github-actions

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Guide unsafe Related to unsafe code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants