RAID1 Disk Setup and Asynchronous Reads by ioeddk · Pull Request #223 · ldos-project/asterinas

ioeddk · 2026-05-05T00:10:28Z

As one of the intermediate PR of #170, merging the following commits:

Updated the QEMU start-up options to use three physical partitions as RAID1 member devices, rather than disk images.
Updated the RAID1 process_read to perform asynchronous reads.
Resolve the long tail latency of the RAID device by creating the RAID1 thread with the real-time scheduler.
Changed the OQueue used in the VirtIO module to the new OQueue.

initialize disk selection policy when starting RAID.

implementation

setting the RAID1 module's thread to use realtime scheduler

arthurp

Mostly nits that should be trivial to fix.

The complex bit is creating and configuring the block devices. See comments below.

arthurp · 2026-05-05T16:33:09Z

-    -drive if=none,format=raw,id=r0,file=./test/build/raid1_0.img \
-    -drive if=none,format=raw,id=r1,file=./test/build/raid1_1.img \
+    -drive if=none,format=raw,id=r0,file=/dev/nvme0n1p1 \
+    -drive if=none,format=raw,id=r1,file=/dev/nvme1n1p1 \
+    -drive if=none,format=raw,id=r2,file=/dev/nvme2n1p1 \


I'm not sure how to handle this given the discussion of multiple configurations above. Do you know off hand when this file is actually used? I think, most of the time, it's actually ignored. If we can understand when this file is actually used we can potentially remove the raid devices entirely and just document that raid isn't configured in the specific configuration.

The configurations here is configuring QEMU if emulating Asterinas on RISC-V. Just deleting them doesn't affect the CI/CD or development for us (we are defaulting to x86_64). But if there are RISC-V emulation in CI/CD it may fail.

OK. If the makefile creates image files by default, it's probably best to just point this at those files.

ioeddk

Addressed minor changes. Will change qemu_args.sh and optimize snafu error handling ASAP.

ioeddk · 2026-05-11T03:39:47Z

-    -drive if=none,format=raw,id=r0,file=./test/build/raid1_0.img \
-    -drive if=none,format=raw,id=r1,file=./test/build/raid1_1.img \
+    -drive if=none,format=raw,id=r0,file=/dev/nvme0n1p1 \
+    -drive if=none,format=raw,id=r1,file=/dev/nvme1n1p1 \
+    -drive if=none,format=raw,id=r2,file=/dev/nvme2n1p1 \


The configurations here is configuring QEMU if emulating Asterinas on RISC-V. Just deleting them doesn't affect the CI/CD or development for us (we are defaulting to x86_64). But if there are RISC-V emulation in CI/CD it may fail.

…g on GH runners, rather than physical partitions

IRQ for OQueue strong observe and converting data writeout on the fly into flush all on inactive.

arthurp

Sorry to nit pick so much. But there are just a lot of things that feel like they haven't been checked or that I think we have discussed before but you haven't addressed. Please read the entire diff yourself again with all previous comments in mind. Let's chat a little about this when we meet today.

Since we have reached a point where nothing is a major issue I can't fix in an already planned next PR, go ahead and do your full pass and then merge.

arthurp · 2026-05-14T15:17:06Z

+# Configure RAID drive sources. Set RAID_DEVICES to a comma-separated list of
+# exactly three existing block devices (e.g.
+# RAID_DEVICES=/dev/nvme0n1p1,/dev/nvme1n1p1,/dev/nvme2n1p1) to pass them directly to the guest.
+# If unset or any device path is invalid, 1GB ext2 image files are used instead.


NIT: In general, I want something to fail if there is invalid data (as opposed to missing), not use a default. This is because the user probably meant to specify something, but did it wrong. If they wanted the default, they wouldn't have specified anything at all. This is something AI does wrong constantly, I don't know if you used it here, but it's worth noting.

arthurp · 2026-05-14T15:23:59Z

+            cp "$RAID_DEV_0" "$RAID_IMG"
+        fi
+    done
+    RAID_CACHE="writeback"


Why is the cache mode different? I'm not saying it's wrong, I'm just curious.

arthurp · 2026-05-14T15:24:16Z

-SSH_RAND_PORT=${SSH_PORT:-22}
+SSH_RAND_PORT=${SSH_PORT:-61541}


Why did this change?

Some fixes, like changing this back to 22, are in some later commits in #170. Since this branch is a cherry-picked first few commits from #170, those places appear to be unfixed.

arthurp · 2026-05-14T15:26:31Z

-#[derive(Clone)]
+#[derive(Clone, Copy, Default, BinarySerde)]
+#[repr(C)]
 pub struct BlockDeviceCompletionStats {
-    /// The latency of the I/O request (time from submission to completion).
-    pub latency: Duration,
+    /// The latency of the I/O request in microseconds.
+    pub latency_us: u64,
    /// The number of outstanding requests at completion time.
-    pub outstanding_requests: usize,
+    pub outstanding_requests: u64,
+    /// The index of the device that produced this stat.
+    pub device_index: u64,


NIT: Aren't we dropping these changes? Once I move to CBOR, Duration works and the derive is different. It doesn't really matter I guess, but I'm going to immediately undo these changes. I thought we discussed this, but maybe I'm wrong, or maybe I said not to worry about it. Sorry, if those are the case.

arthurp · 2026-05-14T15:28:58Z

+                    // Disable IRQs while holding the OQueue's SpinLock inside
+                    // try_strong_observe to prevent deadlock with the IRQ handler
+                    // that produces to the same OQueue (bio completion stats).
+                    let _irq_guard = ostd::trap::irq::disable_local();
+                    o.try_strong_observe()


Is this still needed after #201 landed?

arthurp · 2026-05-14T15:29:15Z

    /// Dispatches a request by type. The RAID-1 device accepts the same BIOs as
    /// any `BlockDevice` and applies RAID semantics underneath.
    fn process_request(&self, request: BioRequest) {
+        // log::info!("Raid1Device process request, type: {:?}", request.type_());


Commented code.

arthurp · 2026-05-14T15:30:41Z

    /// Handles the irq issued from the device
    fn handle_irq(&self) {
-        info!("Virtio block device handle irq");
+        // info!("Virtio block device handle irq");


Commented code.

arthurp · 2026-05-14T15:31:03Z

+    // single disk benchmark
+    // let nvme_device_name = "raid0";
+    // if let Ok(block_device_nvme) = start_block_device(nvme_device_name) {
+    //     let nvme_fs = Ext2::open(block_device_nvme).unwrap();
+    //     let target_path = FsPath::try_from("/raid1").unwrap();
+    //     self::rootfs::mount_fs_at(nvme_fs, &target_path).unwrap();
+    //     info!("[kernel] Mounted NVMe fs at {:?} ", target_path);
+    // } else {
+    //     error!("[kernel] Failed to start NVMe block device '{}'", nvme_device_name);
+    // }
+    // return;
+


commented code.

ioeddk · 2026-05-16T02:32:40Z

All issues addressed.

ioeddk added 4 commits May 4, 2026 19:03

Configure the RAID1 member disks in qemu, and

b26353d

initialize disk selection policy when starting RAID.

Change the RAID1 prcess_read to asynchronous

8bb2e08

implementation

Resolve the long RAID1 tail latency issue by

f1e669a

setting the RAID1 module's thread to use realtime scheduler

Update the RAID1 and VirtIO module to use the new OQueue API

a0693db

ioeddk requested a review from a team as a code owner May 5, 2026 00:10

arthurp reviewed May 5, 2026

View reviewed changes

Setup ORPC data capture for RAID1 IO data

b18350d

ioeddk commented May 11, 2026

View reviewed changes

ioeddk added 7 commits May 11, 2026 03:43

minor changes

ac3457a

Configured the RAID member devices to be partition images when runnin…

d4f768f

…g on GH runners, rather than physical partitions

Derive the VirtIODevice Error with snafu

67829b1

Fixed the build with snafu error

c1ce8a0

Fix the data capturing hanging problem by disable

4951665

IRQ for OQueue strong observe and converting data writeout on the fly into flush all on inactive.

Change synchronous flushing to asynchronous flushing

7825c93

Update Selector Policy to use new OQueue API

b6f40da

arthurp reviewed May 13, 2026

View reviewed changes

Comment thread tools/qemu_args.sh Outdated

Comment thread tools/qemu_args.sh

ioeddk added 4 commits May 14, 2026 02:26

Removed redundant data collecting code for RAID1.

ead8839

Reallocate PCIe lane for data capture file

15f50f0

Using comma separated list as RAID member devices

acc9afd

Update baseilne Asterinas RAID to use roundrobin

6819de0

ioeddk force-pushed the yingqi/raid1_merge/setting_up_raid_qemu branch from 508dd72 to 6819de0 Compare May 14, 2026 02:26

arthurp approved these changes May 14, 2026

View reviewed changes

ioeddk mentioned this pull request May 15, 2026

Non-deterministic Heap Allocation Error #226

Open

ioeddk added 4 commits May 15, 2026 02:42

Reverted Changes to Data Capture modules to comply main

26e9189

Updated qemu_args.sh

240a28f

Let Latency tracking in Block Device stack based on Duration, not u64.

024a9ee

Reformatted

b63bd91

ioeddk merged commit 13484de into main May 16, 2026
22 checks passed

		SSH_RAND_PORT=${SSH_PORT:-22}
		SSH_RAND_PORT=${SSH_PORT:-61541}

Conversation

ioeddk commented May 5, 2026

Uh oh!

arthurp left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ioeddk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

arthurp left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ioeddk commented May 16, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants