Researchers Cut DeepSeek AI Memory Use By 75%

Chinese AI lab DeepSeek‘s models could get far cheaper to run after researchers found a way to cut one big memory cost by about 75% with almost no loss in quality.

Key Takeaways

4-bit tables can cut roughly 75% of n-gram table memory while preserving more than 99% of the validation-loss benefit
DeepSeek-V4.1-Flash allocates 196 billion parameters to its Engram N-Gram Memory
The authors tested lower-precision storage across nanochat, Llama and GPT-2 style models trained from scratch
Four-bit storage uses four bits instead of larger numerical formats such as 16-bit or 32-bit precision

The paper said recent open-weight models have leaned on n-gram embedding tables to store short token patterns, but In Engram N-Gram Memory, those structures can become huge.. It used DeepSeek as a concrete example, stating that DeepSeek-V4.1-Flash allocates 196 billion parameters to its Engram N-Gram Memory, a scale large enough to turn one subsystem into a cost problem on its own.

The authors also tested lower-precision storage across nanochat, Llama and GPT-2 style models trained from scratch and found 4-bit lookup memory kept nearly all the quality lift that n-gram memory brings while using only a quarter of the footprint, enabling reinvestment into larger memory budgets in some settings.

DeepSeek 4-Bit Tables Cut A Real AI Cost Center

Memory capacity, not just raw compute, is a binding constraint for many model builders, and parameters stored at lower precision take less space in RAM or VRAM, which cuts serving cost, reduces power draw at inference, and can widen the set of chips and deployment settings where a model remains practical.

The DeepSeek example is telling because its allocation of 196 billion Engram parameters is concrete capital spending that signals priorities, when a lab commits that much parameter budget to a subsystem, it indicates the component materially affects cost per unit of inference and overall supply constraints for deployment hardware.

Also Read: Alibaba's Qwen Slashes Image Model To 8 Steps, Skips Quality Benchmarks

From Full Precision To Leaner Engram N-Gram Memory Storage

This paper demonstrates those gains do not always require full-precision storage, shifting the engineering question from whether to use n-gram memory at all to how aggressively it can be compressed before quality slips too far.

The experiments on DeepSeek, nanochat, Llama and GPT-2 style models provide concrete evidence that lower-precision n-gram tables can preserve validation-loss benefits while reducing memory footprint.

How 4-Bit Tables Change Open Models

Four-bit storage means each stored value uses four bits instead of a much larger numerical format such as 16-bit or 32-bit precision. Researchers pack the same memory into far less space by accepting noisier numerical detail, and the paper shows that for n-gram memory For Engram N-Gram Memory, the reduced precision hurts surprisingly little..

For open-weight developers working outside the richest labs, Using 4-bit Engram N-Gram Memory representations can shrink a costly component. without rewriting the whole architecture, lowering the hardware pressure (VRAM capacity and power) per inference and the cost per served token.

The key practical question is whether labs will reflect this compression in production releases, capital spending like DeepSeek’s Engram allocation suggests where firms actually place their bets, and here the spending indicates these subsystems are prioritized despite their hardware cost.