The hardware moved on. Most software Didn’t.

Separating storage from compute is the most repeated lesson of the last decade of systems design, and it earned the repetition: storage scales on its own invoice, compute comes and goes, and durability becomes somebody else’s problem. For bulk scans it holds. A column store streaming a terabyte barely notices whether the bytes came from a bucket or a drive.

It stops holding where one read decides the next. A transaction commit, a graph walk and a nearest-neighbor search each issue hundreds of dependent reads, each waiting for the last, so a round trip of milliseconds becomes seconds per query. Those workloads belong beside the compute, with one copy of the data on local drives, in RAM or in VRAM, reached through direct I/O and not over a network. Once the bytes are local, the bottleneck moves into the core.

Trusting the compiler is the right default, and it won for good reasons: for scalar control flow, clang and GCC schedule better than any hand, and portable source outlives any one chip. Two decades of flat clocks were absorbed mostly by compilers and runtimes getting smarter.

Auto-vectorization stops at the loop it can prove. Mixed-precision dot products, masked substring search, AES-based hashing and predicated edit distances are different algorithms, not differently scheduled ones, and every extension spells them out as one intrinsic per type, width and mask. Those kernels are written by hand, one per ISA generation, and chosen at runtime. The same gap repeats across the PCIe bus, where most of the box’s compute now sits.

Keeping the engine on the CPU and the dataset in RAM was right for a long time: GPUs were for training, VRAM was small, PCIe was slow, and a pointer-chasing index wanted a general-purpose core. Most storage engines and vector indexes still assume it.

In a current node most of the rated watts are in the accelerators and most of the capacity is in the drives, with the CPU and its RAM between them as the narrowest tier. A CPU-only engine idles the compute; a RAM-only engine caps the data at terabytes. So the graph walk belongs in VRAM, the merge of sorted runs on the GPU, and the kernels on NVIDIA, AMD and Apple devices beside six CPU families. The problem is no longer the number of devices but the copies between them.

When a profile shows malloc, swapping in jemalloc or mimalloc is the right first move, and for allocation-heavy services it pays. Rewriting the hot loop is the right second. Both act on the compute third of the profile.

The other two thirds are memcpy, page faults, cross-socket stalls and waiting, and a faster allocator only makes the next copy cheaper to start. We design the path so the bytes stop moving: io_uring and SPDK read into registered buffers, threads and memory stay on one NUMA node, and pinned buffers or GPUDirect Storage land drive reads in VRAM without a staging copy. A path this direct also removes the safety nets that catch mistakes, which is where testing has to change.

Memory safety is worth the rewrite: a borrow checker retires a whole class of crashes and exploits, and for most services that is the class that pages someone at night. It is the right floor.

It says nothing about the bugs that survive it. A safe program can still leave a graph region no search reaches, accumulate numerical error, publish half a commit, or wrap an address past 2⁶⁴ and keep running. Each needs its own proof: audit reachability from landmarks, check every kernel against extended-precision references, model-check rings and commits in Spin, and schedule with miniature index types to hit every boundary. Those races are decided in nanoseconds, six orders below where a cluster test suite looks.