Disaggregated serving is now the default answer to the oldest problem in LLM inference: prefill and decode want different hardware, so stop making them share it. Run prompt processing on one pool of GPUs, token generation on another, and tune each independently. The idea is settled. What's still a live decision — and where a lot of production stacks quietly bleed latency — is the part nobody draws on the architecture diagram: the wire between the two pools.

Because when you separate the phases, you separate the KV cache from the GPUs that need it. A long prompt's cache is gigabytes, and it has to land in the decode worker's VRAM before the first token can appear. Move it slowly, or move it through host memory, and you've reintroduced as latency exactly the delay disaggregation was supposed to remove. Two projects own this layer in mid-2026, and the most useful thing to understand first is that they are not the same kind of thing.

NIXL is a pipe, not a platform#

NIXL — the NVIDIA Inference Xfer Library, open-sourced at GTC 2025 — is deliberately small. You get a nixl_agent, a way to register the memory regions you'll move, transfer descriptors that say this cache, from here, to there, and a list of interchangeable backend plugins that do the actual moving. UCX handles RDMA over InfiniBand and RoCE. GDS reaches GPUDirect Storage. There's POSIX for the filesystem, OBJ for S3-compatible object stores, plus AZURE_BLOB, HF3FS, LIBFABRIC, GPUNETIO — eleven backends as of the July 8 v1.3.1 release. One of them is Mooncake.

That last detail is the tell. NIXL doesn't decide where your cache lives, how long it's kept, or whether a prefix gets reused. It moves bytes between two endpoints as fast as the fabric allows and hands the policy questions back to you. That's exactly why it sits at the center of NVIDIA Dynamo and llm-d: those stacks bring their own routing and scheduling and want a transport that stays out of the way. If you're building your own orchestration, NIXL is a transport primitive, not an opinion.

Mooncake is a platform that happens to include a pipe#

Mooncake comes from the other direction. Moonshot AI built it to serve Kimi, and it's a full KV-cache-centric architecture: prefill and decode clusters, plus a distributed pool that scavenges idle CPU, DRAM, and SSD across the fleet to store caches. Its Transfer Engine is genuinely fast — up to 87 GB/s on a 4×200 Gbps RoCE network and 190 GB/s on 8×400 Gbps — and speaks TCP, RDMA, EFA, NVMe-oF, NVLink, CXL, and Ascend transports through one interface. But the engine is the floor, not the point. The Mooncake Store is what you're actually buying: a shared cache tier that lets a prefix computed for one request be reused by the next, across nodes, without recomputation.

NIXL asks how do I move this cache? Mooncake asks why am I recomputing this cache at all? Those are different questions, and only one of them is really about transport.

That reuse story is why Mooncake shows up in stacks that have nothing to do with Kimi — SGLang, vLLM, TensorRT-LLM, vLLM-Ascend, LMCache, and NIXL itself all integrate its Transfer Engine. It's become a de facto interop layer for KV cache in the open-source serving world.

The choice, stated plainly#

Because NIXL can drive Mooncake's engine as a plugin, "NIXL vs Mooncake" is close to a category error. The decision that actually matters is how much of the cache-management problem you want to own.

If you're assembling Dynamo or llm-d, or writing your own scheduler, you want a neutral transport and NIXL is the native path — it moves the cache and lets your logic decide the rest. If you want cross-request prefix reuse and a managed cache pool without building distributed storage yourself, Mooncake's Store is a large amount of hard infrastructure you get for free, and its engine is fast enough that you rarely reach for anything under it. In vLLM you can try either behind --kv-transfer-config, swapping NixlConnector for MooncakeConnector without touching your model code.

And the answer worth saying out loud: on a single node, or under modest load, you need neither. The transfer backend is a problem you earn by disaggregating — and disaggregation only pays past a certain scale. Below that line, the fastest KV-cache transfer is the one you never have to make.