Mining perf test

[ping-ke]

When testing es-node in Windows system, it was found that the mining power of es-node was very low, as shown in the table below.

	Disk Type	Mining Power	IO Read/s	CPU Usage
Local SSD E with BitLocker	NTFS	3.30%	22MB	< 10%
Local SSD F	NTFS	3.70%	26MB	< 10%
external SSD F	NTFS	4.00%	26MB	< 12%
external SSD F	exFAT	3.90%	24MB	< 10%

Create this issue to see why and how to increase the speed.

[ping-ke]

Running IO random read test using the following command:

diskspd.exe -d30 -b<block size> -o32 -t8 -r -w0 -L -h .\es-data\shard-0.dat

Read Block Size	NTFS (MB)	NTFS with BitLocker (MB)
1k	229	114
4k	869	453
8k	1629	751
16k	2677	1184
32k	3234	1651
64k	3456	1904
128k	3516	2008

When the miner is doing sampling, the random read size is 4KB each time. Although each sample only reads 32B, Windows reads a 4KB sector each time, so it is calculated as 4KB here. Using the random read test results with a block size of 4KB (453MB/s, 869MB/s) as a comparison, our read speed during sampling (24MB/s) is only 5%/2.7% of the upper limit.

Therefore, if the disk can be better utilized, our ming power should be able to increase significantly, even up to 100%.

[ping-ke]

Issue: Windows Mining Power Lower Than Expected

Machine	Local Laptop	AX101 (65.109.63.154)
System	Windows	Ubuntu
CPU	24 core	32 core
Memory	64G	128G
Disk	CT1000P3PSSD8 / BitLocker (no AES-NI)	SAMSUNG MZQL23T8HCLS-00A07 + RAID 0
IOPS 4KQD32	600k / 300k	1,000k IOPS × 2
FIO	276,224	2,247,168
Mining Power	3.4% (24 Threads)	366% (64 Threads)
Mining IOPS	5,632	585,984
Used IOPS	1.8% / 2%	29% / 26%

---

Root Cause

By default, Windows uses synchronous I/O.
Even if multiple goroutines perform ReadAt() in parallel, the underlying implementation still calls ReadFile() synchronously.
s.File -> f.pread -> f.pfd.Pread()

👉 Result:

• Threads A and B appear concurrent but actually execute sequentially.
• This prevents full utilization of the multi-queue parallel I/O capability of NVMe SSDs.

---

Solutions

1. Multiple Goroutines + Independent File Handles + ReadAt

This is the simplest and most reliable approach (recommended).

Each goroutine opens its own file handle using os.Open, then performs ReadAt.
Windows can schedule I/O operations across multiple handles concurrently.

Advantages

• Simple and robust
• Each goroutine uses an independent handle
• ReadAt avoids locking conflicts
• Fully utilizes SSD/NVMe parallelism

Test Result
72 threads → 101k IOPS → Mining power: 62.1%

---

2. syscall.CreateFile + FILE_FLAG_OVERLAPPED

Using the FILE_FLAG_OVERLAPPED flag enables asynchronous I/O.
You must align buffers properly and use GetOverlappedResult to check completion and read results.

Pros

• Enables true asynchronous I/O

Cons

• More complex programming model
• Must manually handle completion status and byte counts

Test Result
24 threads → 70k IOPS → Mining power: 43%

---

3. FILE_FLAG_OVERLAPPED + FILE_FLAG_NO_BUFFERING

Adding the FILE_FLAG_NO_BUFFERING flag disables system caching.
When using this mode:

• Buffers must be aligned to the disk sector size (usually 4K)
• I/O bypasses system cache, directly hitting the NVMe controller
• Allows true asynchronous overlapped I/O with maximum hardware parallelism

⚠️ Drawbacks

• Requires 4096-byte aligned buffers
• Must use direct Windows API calls (less convenient in Go)

---

Summary

Method	Description	Complexity	Best Result	Mining Power
Multiple goroutines + ReadAt	Simple, concurrent I/O using multiple file handles	★☆☆	101k IOPS	62.1%
FILE_FLAG_OVERLAPPED	True async I/O	★★☆	70k IOPS	43%
OVERLAPPED + NO_BUFFERING	Full async, uncached I/O	★★★	TBD	TBD

---