Md. Asif Uddin

Marginalia XV

DeepSeek V4

modelDeepSeek-V4-Pro and V4-Flash (April 2026)

Source

Cost per token at a one-million-token contextTwo bars per model. Compared with the previous generation, V4 needs about twenty-seven percent of the per-token inference operations and ten percent of the key-value cache at a one-million-token context.at 1M context, relative to V3.2V3.2FLOPs100%KV cache100%V4FLOPs27%KV cache10%Compressed Sparse Attention interleaved with Heavily Compressed Attention. Everyone quoted the 1.6 trillion.The number that matters is the 10%.
Fig. — — At a one-million-token context, roughly a quarter of the per-token operations and a tenth of the key-value cache. Everyone quoted the parameter count.

Everyone quoted the 1.6 trillion. The number that matters is 10%.

At a one-million-token context, DeepSeek-V4-Pro needs roughly 27% of the per-token inference FLOPs and 10% of the KV cache that V3.2 needed. That’s the release. The parameter count is downstream of it.

Quick shape. Two mixture-of-experts models, April 2026, MIT license, weights on Hugging Face. V4-Pro is 1.6T total with 49B active per token. V4-Flash is 284B total with 13B active. Both default to 1M context and 384K max output. Three reasoning modes: non-think, think high, think max.

Three architectural changes, and only one of them is about size.

Hybrid attention. Rather than one mechanism everywhere, V4 interleaves Compressed Sparse Attention and Heavily Compressed Attention. CSA compresses the sequence, then runs sparse selection over the compressed blocks. HCA compresses harder and attends densely. That stack is where the 10% comes from.

Manifold-constrained hyper-connections. The residual stream gets constrained onto the manifold of doubly stochastic matrices, stabilizing signal propagation through a very deep model without surrendering expressivity.

And the Muon optimizer in place of AdamW, at 1.6T scale.

The headline result is 80.6% on SWE-bench Verified for V4-Pro-Max, the best open-weights score, level with Gemini 3.1 Pro.

But read DeepSeek’s own model card. It says Flash-Max reaches comparable reasoning to Pro given a larger thinking budget, and trails mainly on pure knowledge and the hardest agentic work. 284B against 1.6T, and the gap is narrower than the ratio suggests. Same lesson as EVA-CLIP. Scaling is real and the curve is flat.

Now the parts that don’t make the launch thread.

You will not run Pro. The download is around 865GB. Flash is roughly 160GB, which is at least a conversation. Open weights are real here, and open is not the same as runnable.

Thinking mode defaults on, and reasoning tokens bill as output tokens. Think Max can emit an order of magnitude more tokens than non-think on a hard problem. Your bill scales with a setting, not with your prompt.

The expert count isn’t disclosed for either variant. MIT weights, undisclosed routing topology.

And most serverless hosts quantize activations to fp8, so whatever your router calls DeepSeek V4 is not the reference weights. If you’re benchmarking, check what you’re actually being served.

One thing to credit, after Qwen3.8. This is MIT. Both models, no revenue clause, no access gate, no approval form. That is what open weights are supposed to mean, and it’s rarer than it should be.