Four Nvidia DGX Spark Boxes Serve DeepSeek To 16 Users At Once

Four small Nvidia DGX Spark desktop boxes just served the open DeepSeek V4.1 Flash model to 16 different users at once, at about 24 tokens a second each, with no server rack, no liquid cooling and no cloud bill.

Key Takeaways

A custom, open-source build of DeepSeek V4.1 Flash served 16 different users at 380 tokens a second in total on four linked DGX Spark boxes, and 438 tokens a second when 16 copies of one request ran together
A single user got 80 tokens a second in chat and 138 tokens a second when writing code
The setup handled a 1 million token context and read prompts at 10,000 tokens a second
It beat SGLang TP4 on the same boxes by 18% to 25% for one user, with about 1.9x faster prompt reading

What The Test Showed

A developer posting as Picomoorhead open-sourced the build, a serving engine written for this one model on four DGX Spark units. Tokens are the word chunks AI models read and write, so 80 tokens a second is faster than anyone can read.

Industry analyst Patrick Moorhead, founder of Moor Insights &amp. Strategy, shared the numbers and said they answer the main complaint about DGX Spark. “The knock on DGX Spark was that it’s a developer box, not an inference box,” he wrote. “This open-source DeepSeek V4.1 Flash custom build on four of them says otherwise.”

Four DGX Spark boxes running DeepSeek V4.1 Flash	Result
Chat, one user	80 tokens/sec
Code, one user	138 tokens/sec
16 different users at once	380 tokens/sec (~24 each)
16 copies of one request	438 tokens/sec (~27 each)
Prompt reading speed	10,000 tokens/sec
Context window	1 million tokens
Vs. SGLang on the same boxes	18-25% faster, ~1.9x prompt reading

Also Read: Amazon Adds Nvidia Chip Sharing To Cut AI Serving Costs

Software Was The Bottleneck

Moorhead said the result came from software, not new hardware. “The hardware was never the limiter. The software was,” he wrote, adding that an engine built for one model on one setup beats general-purpose serving tools.

The whole rig is four desktop boxes and a 200G network switch. Moorhead called it “the template for departmental AI,” meaning open models from DeepSeek, small Nvidia boxes and inference software written by the open-source community.

Why It Matters For Companies

Nvidia pitched DGX Spark as a desk-side machine for developers to build and test AI models, while serving answers to many users was left to data center GPUs or the cloud. Serving 16 people at once on four desktop units points to a middle option for teams that want to keep sensitive prompts in house or cap cloud spending.

The catch is that this engine is custom-built for one model on one setup. Teams that switch models or hardware would need similar tuning to get the same speed.

What To Watch Next

The X post said image support was coming soon, and the GitHub README says images work through the normal OpenAI API. The next test is whether other teams can repeat the numbers with the open-source code, and whether similar engines appear for other open models.