Feyd89/Qwen3.8-Flash-Next-GSQ-RCO-capy-flash

Qwen3.8-Flash-Next «64 GB edition» for capy-flash

Ready-to-serve artifact of Qwen3.8-Flash-Next (MoE 125B-A6B) for capy-flash v0.7 or newer, for machines with 64 GB of RAM: the routed experts are ISTA-DASLab's GSQ-RCO IQ3_S release, copied verbatim: a mixed recipe averaging ~3 bits (gate/up in IQ2_S / IQ3_XXS / IQ3_S / IQ4_XS depending on the layer, down in IQ4_NL / Q2_0), which needs 48 GiB of page-locked RAM instead of the 65 GiB of the NVFP4 experts. Everything else (attention indexer, Gated DeltaNet, hyper-connections, PLE, re-distilled MTP draft head) is the same as in Feyd89/Qwen3.8-Flash-Next-NVFP4-capy-flash. On our machine it is also the fastest way to run Flash-Next: smaller experts mean more of them fit in VRAM.

What is inside

part	format
routed experts (48 layers x 512)	the GGUF blocks of ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF (IQ3_S release) copied verbatim (no re-quantization), laid out per expert
MTP experts	IQ4_NL, re-encoded by us from the NVFP4 MTP experts (the GGUF has no MTP layer)
attention indexer, GDN, hyper-connections, PLE, MTP dense part	as in the NVFP4 capy-flash artifact (NVIDIA's checkpoint, MTP head re-distilled on capy-flash's output)
*.draft_shortlist.txt	MTP draft vocabulary shortlist, picked up automatically next to the artifact

Check the download with sha256sum -c SHA256SUMS.

Run

Requires capy-flash v0.7.0 or newer (v0.7.1 for one GPU), Linux (or WSL2 / Docker on Windows), CUDA 13.1, one or two sm_120 GPUs with 16 GB (RTX 50 series), a CPU with AVX-512 VNNI or AVX2 (Zen 2/3, Intel client: picked at startup) and a 64 GB machine (~56 GiB free: 48 GiB of experts in RAM). On a 64 GB machine keep the RAM caches small:

hf download Feyd89/Qwen3.8-Flash-Next-GSQ-RCO-capy-flash --local-dir qwen3.8-flash-next-64gb
cd qwen3.8-flash-next-64gb && sha256sum -c SHA256SUMS && cd ..
ninfer-serve qwen3.8-flash-next-64gb/qwen3_8_flash_next_gsqrco_mtpB.ninfer --profile flash-next \
  --host-kv-mib 1024 --host-state-slots 2 --model-id capy-flash
# one GPU:
ninfer-serve qwen3.8-flash-next-64gb/qwen3_8_flash_next_gsqrco_mtpB.ninfer --profile flash-next-1gpu --model-id capy-flash

Tested inside a 64 GiB memory cgroup without swap: starts in 18 s, peak RSS 55.2 GiB, no paging. With more RAM the default --profile flash-next caches (4 GiB of KV, 8 states) are fine. Windows/WSL2 with a page-locked limit: the expert layers that don't fit stay in pageable RAM automatically (--host-pinned-max-gib to set it by hand).

Chat template updated on 23/10/2026: this artifact now embeds the maintained NInfer Qwen3.8 template (developer role, Responses instructions, system turns after the first message: what Codex CLI sends); the weights are byte-identical. If you downloaded it earlier, run hf download again (only qwen3_8_flash_next_gsqrco_mtpB.ninfer and its last file change), or patch your copy in place with python3 tools/replace_chat_template.py from the capy-flash repository.

Measured with this artifact (capy-flash v0.7.0)

Ryzen 9 9900X (12 cores), 96 GB DDR5, RTX 5070 Ti 16 GB (one or two; PCIe 5.0 x8, no P2P), full clocks, flags above.

	this artifact, two GPUs	this artifact, one GPU	NVFP4 artifact, two GPUs	Strata 0.1.45 IQ3_S (tuned), two GPUs
LocalMaxxing reasoning-v1 (512 tokens)	239 tok/s	121	172	195
agentic replay (53 turns up to 81k), decode	182 tok/s	104	155	151
cold prefill 4k / 16k / 64k	2,117 / 3,305 / 3,426	1,888 / 2,145 / 2,257	2,426 / 4,470 / 4,726	2,216 / 3,811 / 5,233

One GPU uses 58 GiB of RAM. Use capy-flash v0.7.1 or newer for one GPU: its --profile flash-next-1gpu keeps 8 conversation checkpoints in RAM, so agent sessions resume from the cache (1.9 s median to the first token on the replay).

Quality, paired against NVIDIA's NVFP4 checkpoint served by vLLM on a GB300 (thinking on, greedy): GSM8K 0.982 (330 questions; reference 0.985), MMLU-Pro 0.837 (252 questions; reference 0.849); no difference from the NVFP4 artifact on the same questions.

Provenance

• Experts: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF, IQ3_S files, imported with capy-flash python -m tools.convert.gguf_experts assemble (byte copies of the ggml blocks).
• Everything else: Feyd89/Qwen3.8-Flash-Next-NVFP4-capy-flash (from nvidia/Qwen3.8-Flash-Next-NVFP4).
• GPU kernels for the i-quants derive from ggml (MIT) via fngine / ninfer-all (Apache-2.0); see capy-flash's NOTICE.

License

The dense part is a Derivative Model of NVIDIA's checkpoint, governed by the NVIDIA Open Model License (NVIDIA-Open-Model-License.txt); the expert quantization is ISTA-DASLab's, released under Apache-2.0 (LICENSE-Apache-2.0.txt); the underlying model is Qwen3.8-Flash-Next under the Qwen Community License 1.0 (LICENSE-Qwen-Community-1.0.txt). Attribution in NOTICE. Not affiliated with Qwen, NVIDIA or ISTA.