How not to need ZFS on Windows.
I asked AI to contemplate how I set-up file systems for STORJ in a Windows environment, on platform and file system heavily frowned upon on this forum…here’s what it had to say, and by the way yes, it outperforms any tweaks to ZFS you can think of - who need inodes? LOL.
Since this project is on the edge, and they have almost finished converting to hash log stuff, and my business contract is over, I’m happy to share this information, on how AI sees it, it’s actually very accurate compared to my real world results; this is useful for a lot of applications, and I have never seen such an implementation in the wild previously shared on the internet. Excuse whatever garbage formatting it emitted (cuz I can’t be bothered, but go through it you will learn something very advantageous):
Combining a pre-allocated NVMe MFT with a 4-way striped vSAN hybrid capacity layer delivers 10x to 30x faster small-file I/O compared to a single mechanical HDD, while metadata-heavy operations like directory scans, file listing, and Storj node audits run 50x to 100x faster.
This architecture resolves both major performance bottlenecks of spinning disks simultaneously: metadata seek latency and single-spindle queue congestion.
Why This Architecture Outperforms a Single HDD
1. File Creation & Ingress (15x to 30x Faster)
-
Single HDD: Writing a new 164 KB file requires 3 separate physical head movements on the same disk:
-
Seek to locate/update the
$MFTrecord. -
Seek to update the
$INDEX_ALLOCATION($I30B-tree). -
Seek to write the 164 KB payload data clusters.
At ~12.5 ms per random mechanical seek, a single HDD maxes out at 25–35 file creations per second (latencies around 30–40 ms per file).
-
-
Your Setup:
-
Steps 1 & 2 (Metadata): Executed in microseconds on the NVMe-pinned MFT/Index structures.
-
Step 3 (Data Payload): Hits the vSAN SSD Write Buffer first (sub-millisecond ACK back to Windows) and is later destaged in parallel across 4 separate vSAN disk groups/spindles.
-
Result: Ingress latency drops to 1–2 ms per file, allowing sustained creation speeds of 500–1,500+ files per second.
-
2. Cold Random Reads & Egress (4x to 10x Faster)
-
Single HDD: Reading a cold file requires 2 sequential mechanical seeks (1 for the MFT record to get the LCN data run map + 1 for the file payload), yielding ~50–60 random read IOPS total.
-
Your Setup:
-
The MFT lookup is 100% offloaded to NVMe (0.1 ms latency).
-
The 164 KB payload read is distributed across 4 physical vSAN disk groups. Because vSAN striping (Stripe Width = 4) splits/interleaves data across 4 independent mechanical drives (and utilizes vSAN read cache), 4 physical drive heads can satisfy incoming I/O requests concurrently.
-
Result: Random read latency drops from ~25 ms down to ~3–5 ms, increasing aggregate read throughput to 300–600+ random IOPS (or thousands if payload hits vSAN cache).
-
3. Directory Audits & File Scanning (50x to 100x Faster)
Storj nodes periodically execute heavy directory walks, garbage collection, and file audits across millions of pieces.
-
Single HDD: The drive head must jump back and forth continuously across the disk platter to read
$I30directory B-trees and MFT records. Scanning 24 million files on a single HDD can take 24 to 48+ hours, grinding node performance to a halt. -
Your Setup: Because the entire
$MFTand directory index layout (~132 GiB) sits on NVMe, Windows traverses the entire 24-million-file directory tree purely at flash memory speeds. The physical HDDs are never touched during metadata-only scans.- Result: A full 24-million-file directory scan finishes in minutes to an hour instead of days.
4. Queue Depth Handling & Parallelism
A single HDD can only service 1 I/O operation at a time (Queue Depth = 1). When Storj sends concurrent multi-threaded requests, the disk head thrashing causes latency spikes exceeding 500+ ms.
In your setup:
-
Storage Spaces: Treats the underlying vSAN virtual disk as a fast NVMe-backed volume for metadata.
-
vSAN Layer: Converts Windows block I/O into 4 parallel I/O streams across distinct ESXi hosts and disk groups, effectively elevating usable Queue Depth from 1 to 64+.
Quantitative Metric Comparison
| Performance Metric | Standard Single 7200 RPM HDD | Your Setup (NVMe MFT + 4-Way vSAN Stripe) | Performance Gain |
|---|---|---|---|
| MFT Lookup Latency | 8.5 ms – 15.0 ms | 0.05 ms – 0.2 ms (NVMe) | ~100x Faster |
| 164 KB File Creation Latency | 30 ms – 40 ms | 1 ms – 2 ms (vSAN Cache ACK) | ~20x Faster |
| Max File Ingress Rate | ~30 files/sec (~5 MB/s) | ~800–1,500+ files/sec (~130–240 MB/s) | ~30x–50x Higher |
| Cold Random Read Latency | 20 ms – 25 ms | 3 ms – 5 ms | ~5x–8x Faster |
| Random Read IOPS (164 KB) | ~50 IOPS | 300 – 600+ IOPS (Uncached) | ~6x–12x Higher |
| 24M File Directory Walk / Scan | 24 – 48 Hours | 15 – 45 Minutes | ~50x–100x Faster |
25 cents,
Julio
