[DNM] nvme: keepalive fixups (#3998)

[babayet2]

Backport of #3998

Improves the keepalive tests to actually run servicing, and adds a panic when there is a write to the admin queue in fused mode

[babayet2]

The vmm test changes did not cherry-pick cleanly, so I'm waiting to see how this runs, and marking DNM for now

[Copilot]

Pull request overview

This PR backports the “nvme: keepalive fixups” work, updating OpenHCL servicing tests to validate NVMe fused-keepalive behavior across a service boundary, and tightening NVMe driver restore behavior around keepalive invariants and queue restore error handling.

Changes:

• Reworks the OpenHCL servicing keepalive test to assert fused vs non-fused IO-queue behavior before and after restart_openhcl.
• Updates NVMe driver restore logging and adds a fused-keepalive invariant check during admin-queue restore.
• Fixes queue restore to correctly propagate errors (replacing flat_map(Result<...>) error-dropping behavior with collect::<Result<...>>()?).

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
vmm_tests/vmm_tests/tests/tests/multiarch/openhcl_servicing.rs	Refactors the keepalive servicing test to validate fused keepalive semantics via inspect before/after servicing.
vm/devices/storage/disk_nvme/nvme_driver/src/driver.rs	Improves restore diagnostics, enforces fused-keepalive invariants, and fixes queue restore error propagation.

Comments suppressed due to low confidence (4)

vmm_tests/vmm_tests/tests/tests/multiarch/openhcl_servicing.rs:1354

• fused_updater is initialized to false and never set to true, so the HardwareConfigFaultConfig that spoofs VendorID/DeviceID will never become active. That would prevent the device from entering fused keepalive mode, making the assertions later in the test meaningless.

        let mut fused_updater = CellUpdater::new(false);
        // Spoof the vendor and device ID for a device that requires fused keepalive
        let fused_fault = FaultConfiguration::new(fused_updater.cell()).with_hardware_config_fault(
            HardwareConfigFaultConfig::new()
                .with_vendor_id(0x1414)
                .with_device_id(0xb111),
        );

vmm_tests/vmm_tests/tests/tests/multiarch/openhcl_servicing.rs:1335

• The test redefines flags a second time (lines 1344-1346), shadowing the initial flags and likely causing an unused-variable warning/error for the first binding. This also makes it unclear which flags are used for the restart_openhcl call.

This issue also appears in the following locations of the same file:

• line 1347
• line 1348
• line 1441

    let mut flags = config.default_servicing_flags();
    flags.enable_nvme_keepalive = true;

vmm_tests/vmm_tests/tests/tests/multiarch/openhcl_servicing.rs:1442

• with_keepalive_fault_updater / no_keepalive_fault_updater are still used here, but their definitions were removed earlier in the refactor. Additionally, the NVMe controller configs above still reference with_keepalive_fault_config / no_keepalive_fault_config which are no longer defined. As-is, the test will not compile; either reintroduce those per-device FaultConfiguration values or remove the remaining references and wire up the new fused/normal fault configs instead.

    with_keepalive_fault_updater.set(true).await;
    no_keepalive_fault_updater.set(true).await;

vmm_tests/vmm_tests/tests/tests/multiarch/openhcl_servicing.rs:1353

• The new fused_fault (hardware config spoof for fused keepalive) is constructed here, but the NVMe controller configs later still use the old *_fault_config variables. Without wiring fused_fault into the fused controller's fault_config (and providing a separate empty FaultConfiguration for the normal controller), the test setup is incomplete and will fail to compile / exercise the intended behavior.

    let run_vm = async || -> Result<PetriVm<OpenVmmPetriBackend>, anyhow::Error> {
        let mut fused_updater = CellUpdater::new(false);
        // Spoof the vendor and device ID for a device that requires fused keepalive
        let fused_fault = FaultConfiguration::new(fused_updater.cell()).with_hardware_config_fault(
            HardwareConfigFaultConfig::new()
                .with_vendor_id(0x1414)
                .with_device_id(0xb111),

[babayet2]

Closing, opened #4131