deepseek-ai/DeepSeek-V4.1-Flash

Overview

DeepSeek-V4.1 is a vision-language Mixture-of-Experts model: 552B backbone parameters plus 196B of Engram memory, 8B active per prompt token and 16B per output token, 40 transformer layers at hidden size 5120, with a 32-layer ViT and aligner in front of the text stack. Six things distinguish it from V4:

• Causal encoder-decoder. The 40 layers split into a 20-layer encoder and a 20-layer decoder, and the decoder's global KV is projected from the encoder's final hidden state instead of being computed layer by layer. Prefill therefore only runs the encoder half, which is where the 8B-prefill / 16B-decode split comes from.
• Two-tier sparse attention. Every layer attends over a 128-token sliding window. Layers that carry a compression ratio add compressed KV latents reaching further back, pooled by a learned softmax gate. A small side attention — the indexer, inherited from V3.2-Exp — scores those latents and keeps the best 512 per query, pre-filtered by a candidate stage that selects 2048 blocks of 8. Only four layers (2, 8, 14, 20) actually compress their own KV; the rest read that cache. V4.1 uses only compression ratios 1 and 2, where V4 used 4 and 128.
• Engram n-gram memory. Layers 1 and 14 each own a hash table of ~384M rows x 256 dims, looked up by 2-, 3- and 4-gram hashes of the input and written into the residual stream through a learned gate. These two tables alone are 196.6B parameters (~183 GiB) — plan capacity for them, they dominate everything except the experts.
• Hyper-Connections. The residual stream is carried as 4 parallel copies; each sublayer derives its own pre/post/combine coefficients from the stream, with the combine matrix made doubly stochastic by 20 Sinkhorn iterations.
• DSpark draft head. Three stages (128 routed experts each, 3 activated) draft a block of 5 tokens, reading the attention input of layers 37-39, with a Markov bias head and a confidence head on the last stage.
• Mixed MXFP4/MXFP8 checkpoint. Routed expert weights are MXFP4; everything else is MXFP8 block-quantized, UE8M0 scales throughout. Embedding and LM head are BF16.

Routing is 6 of 384 experts per token plus one shared expert, scored with sqrtsoftplus and a noaux_tc bias — with a separate routing bias for tokens inside an image span, so vision and text tokens do not compete for the same experts.

Context length

1,048,576 tokens, reached by YaRN with factor 16 over a 65,536-token training window. Compressed KV rotates at its own RoPE theta (160,000) because one latent stands for several tokens, so its positions are further apart than the raw stream's.

Images

Images enter the prompt as <|deepseek_image|> spans; every position in the span carries the image token id, and the per-position role (start / newline / end / image) comes from the processor. The vision tower is a 32-layer ViT at hidden size 1024, patch 14, with a 3x downsampling aligner and a cap of 1024 tokens per image (minimum 295,936 pixels). There is no limit on images per prompt. Merged embeddings enter the text model as inputs_embeds, before the hyper-connection stream expansion, while raw token ids still flow through so the router can apply the image routing bias.

Tick Encoder parallel to run the ViT data-parallel (--mm-encoder-tp-mode data) instead of tensor-parallel: at 32 layers / hidden 1024 the encoder is small enough that TP communication costs more than it saves, which can significantly reduce TTFT for multi-image requests. It is mutually exclusive with Text only.

Reasoning and tool calling

Two thinking modes and a numeric reasoning budget rather than discrete tiers. Send them as chat_template_kwargs:

key	values
thinking / enable_thinking	boolean; if you send both they must agree
reasoning_effort	low (25), high (50), xhigh (75), max (100), or an integer 1-100

The top-level OpenAI reasoning_effort field also works, where "none" turns thinking off. "minimal" and "medium" are rejected — they are not part of this model's set.

Note that the DeepSeek API maps its low / high / max tiers to 50 / 75 / 100, while the open-source prompt encoder (and therefore vLLM) maps low / high / xhigh / max to 25 / 50 / 75 / 100. A request tuned at high on the API runs at effort 50 here; pass an integer if you need parity.

With both keys unset, thinking is ON at effort 50. The do-nothing config is the most verbose one, so a request with a small max_tokens spends its budget on the trace and returns empty content with finish_reason=length. That reads as a broken model and is not. Either pin thinking: false or give the budget room.

In thinking mode the budget is rendered into the prompt as a Reasoning Effort: N prefix on the first turn only.

resp = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-V4.1-Flash",
    messages=[{"role": "user", "content": "What is 17*19?"}],
    extra_body={"chat_template_kwargs": {"thinking": True, "reasoning_effort": 25}},
)

DeepSeek's published numbers use temperature=1.0, top_p=0.95 and effort 100 with max_tokens of at least 256K. The tech report notes that efforts of 60-80 recover most of that accuracy at well under half the tokens, so reserve max for hard tasks.

Tool calls are wrapped in DSML tag blocks rather than JSON fences, and tool output comes back in <tool_result> tags.

Speculative decoding

DSpark is the only speculative method for this checkpoint. V4.1 dropped the MTP module that V3 and V4 trained alongside the backbone. Speculative decoding enables a 5-token block. On NVIDIA it also turns on adaptive verification (see the feature description). On AMD, vLLM currently refuses adaptive verification. The generated command sets enable_adaptive_verification:false and verifies the full block. Two checks in maybe_create_adaptive_verification_manager fail on ROCm: the indexer helper supports_device_cpu_query_lens_mismatch() is False, and DeepseekV41ROCMAiterSparseSWABackend reports AttentionCGSupport.UNIFORM_BATCH rather than ALWAYS. Remove the AMD override only after a serve with the true flag starts. Acceptance depends on the workload, so measure it on your own traffic before sizing a deployment around it. The drafter's experts add roughly 14B parameters to the load.

Serving text-only

Tick Text only to add --language-model-only, which skips the vision encoder entirely. Worth it whenever the workload is text: it drops the ViT and aligner from the load and frees that VRAM for KV cache. It is mutually exclusive with encoder_parallel. The verified GB200 runs (TP4 and 1P1D) were text-only.

Prefill/Decode disaggregation

The Prefill/Decode Disaggregation strategy is the verified 1P1D layout on GB200 NVL4: one tray (4 GPUs) per role, TP4 in each pool, KV handed over through NIXL, fronted by vllm-router --vllm-pd-disaggregation. Both pools disable FlashInfer autotune plus JIT and CuTeDSL warmup via --kernel-config, skip the DeepGEMM warmup (VLLM_DEEP_GEMM_WARMUP=skip), and cap --max-num-seqs at 32. On 8-GPU nodes the same layout becomes TP8 per role.

With Speculative decoding on, DSpark runs in both pools so the transferred KV stays compatible.

Memory

The checkpoint is roughly 511 GB on disk (476 GiB), which breaks down as:

Component	Entries	Stored
Routed + DSpark experts (MXFP4)	557.2B	259.5 GiB
Engram tables (FP8)	196.6B	183.1 GiB
Attention, dense projections, routers (FP8)	7.4B	6.9 GiB
Embedding + LM head (BF16), norms (FP32)	2.0B	3.9 GiB
UE8M0 block scales	23.6B	21.9 GiB

DeepSeek's 552B backbone figure covers the routed experts, attention and embeddings; the Engram tables, the ~14B DSpark drafter and the block scales sit outside it.

vram_minimum_gb: 614 is that total times the schema's 1.2 headroom factor. It fits one GB200 NVL4 tray (768 GB) at TP4, or one 8-GPU H200 node (1128 GB) with room for KV cache.

KV cache is a small share of that budget. The compressed latents are shared across layers and trained to be stored in FP4, and DeepSeek puts the resulting global KV at 890 bytes per token, about a quarter of V4-Flash; by that accounting a full 1M-token prompt holds under 1 GB of global KV, plus a fixed 128-token sliding window per layer. Weights and batch size, not cache, set the capacity limit — still, measure before assuming a given context/batch fits.

Prerequisites

• An image (vLLM 0.30.0+); no pip wheel serves this architecture, so the Install block only offers Docker. On NVIDIA that is vllm/vllm-openai:deepseekv41-flash-0909. On AMD it is vllm/vllm-openai-rocm:nightly-eed1f3d0c6043bd494424a22443ee198dd56f657 — a ROCm nightly rather than the 0909 tag, because 0909 predates vllm-project/vllm#56503, which moves the mHC delayed pre block off the eager Torch reference and onto AITER.
• The Rust OpenAI frontend is selected by default in the command builder. Switch to Python if you encounter unsupported features or compatibility issues.
• Expect a long first load: VLLM_ENGINE_READY_TIMEOUT_S=3600 is set for that reason.
• On AMD, the generated command sets VLLM_USE_BREAKABLE_CUDAGRAPH=1. DeepSeek-V4.1-Flash does not support torch.compile, and the ROCm sparse SWA backend only supports uniform-batch CUDA graphs. Without breakable CUDA graphs, default FULL_AND_PIECEWISE dies at capture.
• On AMD, the generated command passes --moe-backend aiter rather than naming a kernel. Naming aiter_triton_mxfp4_bf16 pins the Triton W4A16 _moe_gemm_a16w4 kernel; the plain name lets vLLM select the Composable Kernel a8w4 experts, which measured 4–9% faster per decode step at 131k context on MI355X, with larger gains at higher concurrency. It also sets AITER_TRITON_LOG_LEVEL=ERROR to drop AITER's per-call "Gluon unavailable" warning, which is constant on gfx950 and dominated the server log.
• On MI355X, Tensor Parallel additionally pins --max-num-seqs 128, the value the verified run served. Left unset, vllm serve defaults to 1024 on GPUs with at least 160 GiB. Raise it if your traffic needs more scheduler capacity than the verified configuration.
• On AMD, ticking Speculative decoding also sets enable_adaptive_verification:false. vLLM currently refuses the true flag: DeepseekV41IndexerBackend.supports_device_cpu_query_lens_mismatch() is False, and DeepseekV41ROCMAiterSparseSWABackend reports UNIFORM_BATCH rather than ALWAYS. DSpark still drafts 5 tokens per round.

Verifying

Serve, then send one text request and one image request — the vision tower is a separate path and a text-only smoke test will not exercise it:

curl http://localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"deepseek-ai/DeepSeek-V4.1-Flash",
       "messages":[{"role":"user","content":"What is 17*19? Return only the integer."}]}'

A correct answer is 323.

References

• Model card
• Technical report
• DeepSeek-V4-Flash recipe — the V4 sibling this diverges from
• DeepSeek-V3.2-Exp recipe — where the indexer first appeared