Nick Mermiges, a lawyer and active member of the r/LocalLLM community, has scaled his home AI server from eight to twelve used NVIDIA Tesla V100 32GB cards. The $9,450 build allows him to run DeepSeek V4.1 Flash with all 15,360 routed experts resident in GPU memory at 4-bit NVFP4 precision. This move eliminates the need to compress experts further or swap them out during inference, a common bottleneck in smaller local setups.
The Memory Math Behind the Upgrade
The expansion was driven by strict memory requirements rather than a desire for raw compute power. According to Mermiges' GitHub write-up, the NVFP4 experts alone require 278.9 GiB across 40 layers. His previous eight-card setup offered only about 254 GiB, forcing him to run experts at 2-bit precision and keep only the most frequently used 38.7 percent in faster NVFP4 format. Adding a third NVLink board with four GPUs brought total HBM2 memory to 384 GiB, providing enough headroom to store the entire expert set uncompressed.
Software Unpacking on Legacy Hardware
A critical technical detail is that the 2018-era V100 Volta chips lack native FP4 or FP8 tensor cores. Consequently, the NVFP4 values are unpacked into half precision in software before computation. Mermiges explains that the performance gains stem from reading fewer bytes from memory, not from faster mathematical operations. Despite this software overhead, he confirms that the expert values match DeepSeek's official release, while other model components remain in compact forms to save space.
Performance Gains and Honest Limitations
Self-reported benchmarks show code generation reaching up to 184 tokens per second on a single stream. The most significant improvement is in latency for long documents; the first code token at 256K context now arrives in 58 seconds, down from 108.4 seconds on the previous build. Prompt reading speeds above 32K context jumped from 2,200 to about 4,500 tokens per second. However, Mermiges notes that decoding speeds do not improve significantly at contexts of 32K and above, and a 256K file still takes 185 seconds to read fully.
Cost Efficiency Against Modern Stacks
The build reportedly outperforms published four-unit DGX Spark stacks in single-request decoding by 15 to 45 percent for code tasks, at roughly half the price of a three-unit Spark setup. Mermiges optimized the hardware by raising the power cap from 150 to 175 watts and lowering the clock cap from 1,522 to 1,452 MHz, resulting in a net performance gain. The setup does face architectural limits, as the third NVLink board sits behind a separate PCIe switch, introducing additional latency on the second hop.
Key Takeaways
- Memory bandwidth and capacity are the primary bottlenecks for local LLMs, not raw compute power.
- Legacy hardware like the 2018 V100 can run modern 2026 model formats via software unpacking.
- Self-reported data shows halved latency for first-token generation on long documents.
- The $9,450 build outperforms some modern enterprise stacks in specific code tasks at half the cost.
The Bottom Line
This project proves that for local LLMs, memory capacity and bandwidth are often the primary bottlenecks, not compute power. By leveraging discontinued silicon and software workarounds, builders can achieve high-end performance metrics at a fraction of the cost of modern enterprise hardware.