You need to agree to share your contact information to access this model
This repository is publicly accessible, but you have to accept the conditions to access its files and content.
Responsible Use Agreement
This model is derived from an abliterated DeepSeek V4 Flash checkpoint with safety refusals reduced or removed. It is provided for controlled local evaluation, red-teaming, security research, and unfiltered assistant experiments. By requesting access, downloading, or using these weights, you accept responsibility for applying appropriate access controls, safety filtering, monitoring, and human review for your deployment.
Prohibited uses:
- Sexual exploitation or endangerment of minors.
- Self-harm or suicide promotion.
- Illegal activity in your jurisdiction, including fraud, harassment, doxxing, or targeting real individuals.
- Generating instructions intended to cause physical, chemical, biological, cyber, or other real-world harm.
- Any use prohibited by the upstream DeepSeek license or applicable law.
The weights are provided as-is, with no warranty. You are solely responsible for your inputs, outputs, deployment controls, and compliance obligations.
Log in or Sign Up to review the conditions and access this model content.
DeepSeek V4 Flash 0731 Abliterated NVFP4
DGX Spark / GB10 users: prefer drowzeys/keys-DeepSeekV4-Flash-GA-0731-Dspark-Abliterated-32-32 for Flame/Ember-style two-Spark deployments. That checkpoint keeps the native DeepSeek/DSpark MXFP4 path and, in our Spark tests, served faster than this NVFP4 conversion. This NVFP4 checkpoint is mainly useful for server-class Blackwell/NVIDIA stacks where the NVFP4 kernels are the intended path, such as RTX PRO 6000-class or data-center deployments. Do not pick this repo for DGX Sparks expecting higher speed.
This is a two-node DGX Spark-ready NVFP4-expert conversion of apetersson/DeepSeek-V4-Flash-0731-Abliterated-FP8. It preserves the upstream Abliterated checkpoint's FP8 attention, dense, shared-expert, and MTP tensors while converting the 33,024 routed-expert weights to NVIDIA's NVFP4 layout with calibrated activation scales.
The name is intentionally shorthand, not a claim that every tensor is NVFP4. This remains a mixed-precision checkpoint:
- routed experts: NVFP4 E2M1 weights, E4M3 block scales, FP32 tensor and input scales;
- attention and other default linear layers: FP8 E4M3;
- shared experts and MTP experts: retained from the source format.
What was converted
The upstream repository is labelled FP8 and declares FP8 as its default
quantization method, but direct tensor inspection shows that its routed experts
are packed MXFP4 E2M1 with per-32-element UE8M0 scales. We therefore used
NVIDIA ModelOpt's lossless MXFP4-to-NVFP4 weight cast rather than dequantizing
and requantizing those weights. Activation input_scale values were measured
with a real two-node calibration run.
Pinned inputs and tooling:
- source revision:
7d02640c72a2c8127f116d3d1933ddfec5e4c0fa; - NVIDIA ModelOpt revision:
2f6e77f14cd80593ddc0dda434f5acfb77a62e7c; - ModelOpt version:
0.46.0rc1.dev13+g2f6e77f14; - calibration hardware: 2x NVIDIA GB10 DGX Spark, model parallel size 2;
- calibration: 128 public prompts, batch size 4, sequence length 512, left padding;
- prompt mix: 32 each from CNN/DailyMail, Magpie Pro MT, OpenCodeReasoning, and OpenMathReasoning;
- frozen calibration JSONL SHA-256:
47cd1af8730e9b6149c4794d83dc152b2925442520675ab642f1ec88d8288f59.
NVIDIA's Nemotron Post-Training Dataset v2 was gated and unavailable on the conversion nodes, so it was explicitly replaced with the public chat, code, and math sources above.
Export results:
8,657,043,456 / 8,657,043,456routed-expert blocks cast losslessly;- 33,024 NVFP4 routed-expert weight tensors across 43 layers;
- 65,982 rank-local amax entries merged;
- 33 score-routed experts received the documented same-projection input-amax fallback because they received no tokens in this calibration sample;
- all 48 output shards independently audited on both DGX Spark nodes;
- zero structural errors and zero on-disk w1/w3
weight_scale_2mismatches; - final Hub audit: 48/48 sequential shards, 175,550,788,904 shard bytes, and 138,365 tensor entries mapped by the index across exactly those 48 shards.
Two-Spark validation
The checkpoint was served with tensor parallel size 2 across two DGX Sparks. The same release and runtime image were independently validated on two Spark pairs: Flame + Ember and Forge + Anvil. The validated profile used:
- a DSpark-enabled vLLM build based on
0.25.2.dev0+g752a3a504.d20260714; - FlashInfer Python/JIT
0.6.15; - native SM121
flashinfer_b12xNVFP4 MoE backend; - 1,048,576-token maximum context;
- NVFP4 DeepSeek MLA KV cache;
- MTP/DSpark speculative decoding with 5 speculative tokens;
max_num_seqs=8,max_num_batched_tokens=16384, and GPU memory utilization 0.87;- CUDA graph capture through 48 tokens.
Measured KV capacity varies slightly with free runtime memory on each pair:
| Spark pair | Shared KV pool | Even share at concurrency 8 |
|---|---|---|
| Flame + Ember | 1,480,519 tokens | 185,064 tokens/session |
| Forge + Anvil | 1,458,004 tokens | 182,250 tokens/session |
The KV pool is shared across active requests, and each figure counts prompt plus generated tokens. The 1,048,576-token setting remains the ceiling for an individual request when the other sessions leave enough pool capacity; it does not mean eight simultaneous 1M-token sessions. For a portable concurrency-8 budget across both validated pairs, use about 182K tokens per active session and leave a little headroom for KV-block rounding and reusable cached prefixes.
Forge + Anvil also passed a live concurrency-8 deployment smoke test: 8/8 simultaneous requests completed and produced all 512 requested output tokens. This short qualification run verifies scheduling and routing; the longer throughput results below remain the reported performance benchmark.
You need a DeepSeek V4/ModelOpt NVFP4-capable vLLM build. A stock Transformers
quickstart is not sufficient for this experimental checkpoint layout. The
validated runtime adds a small vLLM-to-FlashInfer adapter for the exact DeepSeek
activation (swigluoai_uninterleave, alpha 1.0, beta 0.0, limit 10.0) and shares
one geometry-identical B12X wrapper per GPU process across MoE layers. This is a
runtime integration change only; it does not alter checkpoint tensors.
The original four-checkpoint comparison used the earlier four-sequence profile. The current deployed profile was subsequently tuned and rebenchmarked without changing the checkpoint, quantization, activation semantics, speculative-token count, or context ceiling. The backend was then upgraded from CUTLASS to the SM121-native B12X path described below.
Allow roughly five minutes for backend initialization and CUDA graph capture after the weights have loaded on a fresh process. The first high-concurrency request can also include JIT/path-settling overhead; use warmed measurements for steady-state capacity planning.
Abliteration verification
We checked preservation in two complementary ways. First, the upstream ablation manifest and all 36 tensors it identifies were compared before and after conversion; the manifest was byte-identical and every listed tensor was bit-identical. Second, a compact behavioral refusal/steerability suite spanning benign controls, sensitive contexts, and adversarially phrased boundary cases produced 12/12 substantive responses with zero detected refusals. Test prompts and generated text are intentionally not published.
These results are consistent with the upstream ablation having been retained. The behavioral suite is a small verification sample, not a universal guarantee about model behavior.
Benchmarks
All performance measurements below used the same Flame + Ember two-Spark profile. The performance test issued eight serial requests with up to 256 output tokens each, producing 2,048 completion tokens total.
| Checkpoint | Expert/backend | Deterministic | Guided | Output tok/s | p50 | p95 |
|---|---|---|---|---|---|---|
| DeepSeek V4 Flash 0731 | MXFP4 / B12X | 9/9 | 4/8 | 40.09 | 6.435s | 6.539s |
| Rarri NVFP4 | NVFP4 / CUTLASS | 9/9 | 4/8 | 26.50 | 9.705s | 10.100s |
| Abliterated source | MXFP4 experts / B12X | 9/9 | 3/8 | 38.41 | 6.652s | 6.765s |
| This checkpoint | NVFP4 / B12X | 9/9 | 4/8 | 27.90 | 8.827s | 10.332s |
The first three rows retain the original validation-run measurements; the Neko row is the current warm B12X serial measurement. On that basis this checkpoint is 5.3% faster than the separately converted Rarri NVFP4 checkpoint, while it remains 27.4% slower than its own MXFP4-expert source in this serial test. The guided suite improved from 3/8 to 4/8 in this run, but eight attempts are far too few to claim a statistically meaningful quality improvement.
Runtime tuning: same NVFP4 checkpoint
The serving profile was first tuned for the actual two-Spark workload and then the valid SM121 B12X path was enabled. The control used four active sequences, 8,192 batched tokens, CUDA-graph capture 24, and GPU memory utilization 0.80. Both tuned profiles use eight active sequences, 16,384 batched tokens, CUDA-graph capture 48, and utilization 0.87.
| Concurrency | Control CUTLASS tok/s | Tuned CUTLASS tok/s | Shared B12X tok/s | B12X vs tuned CUTLASS |
|---|---|---|---|---|
| 1 | 26.05 | 26.06 | 27.90 | +7.1% |
| 4 | 52.72 | 55.49 | 57.69 | +4.0% |
| 8 | 54.56 | 71.22 | 85.28 | +19.8% |
At concurrency 8, moving from tuned CUTLASS to shared B12X reduced p50 latency from 27.320s to 23.496s and p95 from 31.536s to 26.079s. The B12X concurrency-4 and concurrency-8 values are the mean of two warm passes; concurrency 1 is one warm pass. The first cold B12X passes measured 42.20 tok/s at concurrency 4 and 60.01 tok/s at concurrency 8 while JIT and runtime paths settled, so they are reported separately rather than mixed into the warm comparison. Each request used the same prompt and up to 256 output tokens with EOS ignored.
Final B12X checks produced 9/9 structured-format passes, 4/8 guided passes, and 12/12 substantive behavior-probe responses with zero detected refusals. Prompt and response text are not published. These small samples are regression checks, not statistical accuracy claims.
Raw aggregate measurements are published in
deepseek-v4-flash-0731-abliterated-nvfp4-b12x-shared-2026-08-03.json.
The earlier CUTLASS profile, structural preservation audit, and checkpoint
comparison are retained under benchmarks/.
Why B12X now wins on Spark
NVFP4 is designed for greater numerical fidelity than MXFP4: it uses a finer per-16-value E4M3 scaling hierarchy, while MXFP4 uses per-32-value E8M0 scales. That format advantage does not guarantee that every available runtime kernel is faster. See NVIDIA's NVFP4 format overview.
FlashInfer 0.6.15 has a native SM121 B12X path that can preserve this model's clamped activation. The remaining issue was the vLLM adapter: it did not pass the alpha, beta, and limit arguments through, and it constructed a maximum-size B12X wrapper for every MoE layer. At this model geometry, one wrapper used about 0.76 GiB; duplicating it across the model would demand roughly 46 GiB of extra memory and prevented a full startup.
The validated adapter passes the exact activation parameters and reuses one geometry-identical wrapper per rank. Model layers execute serially on the CUDA stream, so the shared scratch/output storage is safe for the captured execution used here. Standalone dynamic and graph-shape checks were run through 16,384 tokens on both ranks before full model startup. Full initialization, CUDA graph capture, structured checks, guided checks, and behavioral checks then passed.
The older tuned flashinfer_cutlass setup remains a known-good rollback. Marlin
was not scored because this stack warned that it would reinterpret the signed
NVFP4 scale format, while the TensorRT-LLM NVFP4 MoE path remains gated to the
SM100 family rather than DGX Spark's SM121. On this exact W4A4 workload, shared
B12X is the fastest qualified backend without changing activation behavior or
scale interpretation.
The exact runtime bridge and its standalone SM121 geometry check are published
as runtime/patch_vllm_b12x_swiglu.py
and runtime/precompile_b12x_deepseek_v4.py.
Historical loader warning
The earlier CUTLASS validation runs emitted:
w1_weight_scale_2 must match w3_weight_scale_2. Accuracy may be affected.
The warning appeared for both community NVFP4 checkpoints only after vLLM loaded and fused the weights. Full on-disk audits found every w1/w3 pair exactly equal, so this is recorded as a CUTLASS loader/fusion caveat rather than an on-disk checkpoint mismatch. It was not present in the final B12X service logs.
Attribution
- Base model: DeepSeek-AI DeepSeek V4 Flash 0731
- Abliterated source: apetersson/DeepSeek-V4-Flash-0731-Abliterated-FP8
- Quantization/export tooling: NVIDIA TensorRT Model Optimizer
- Published conversion:
neko-legends
The upstream ablation changes are preserved; this conversion does not add a new safety or alignment claim. Review the upstream model card and license before deployment.
- Downloads last month
- 1,777
Model tree for neko-legends/DeepSeek-V4-Flash-0731-Abliterated-NVFP4
Base model
deepseek-ai/DeepSeek-V4-Flash-0731