A researcher has patched llama.cpp to run DeepSeek V4 Flash with a full 1-million-token context window on a single RTX 5090, slashing VRAM requirements from an estimated 256 GB to roughly 31 GB. The bottleneck stemmed from the model's DSA lightning attention indexer, which lacked a CUDA implementation and wasn't wired into llama.cpp's computation graph. An upstream pull request addressed the wiring but left the GPU path unfinished. The researcher, posting as u/da_dragon321, implemented the missing CUDA kernel and integrated the indexer, enabling practical long-context inference on consumer hardware.
Benchmarks confirm the gains. At 256K context, prefill throughput jumps from 56 to 263 tokens per second while decode holds steady at 14 t/s. The 1M context configuration runs at 159 t/s prefill and 13.7 t/s decode within 31 GB of VRAM. Needle-in-haystack tests validated retrieval accuracy at depths of 100K, 512K, and 1M tokens.
The patch is available on a public branch with build instructions, though no prebuilt binary exists yet. Only the RTX 5090 has been tested, but the approach should generalize to other Blackwell GPUs. For developers and researchers, this unlocks local experimentation with frontier context lengths previously reserved for multi-GPU clusters.
