52 full-attention layers versus 16: mapping Nanbeige and Qwen
Nanbeige4.3-33B and Qwen3.8-27B sit near the same parameter scale, but their released decoder configurations describe very different attention stacks. We made an interactive layer map so you can inspect that difference instead of reducing both models to a single size number.
| Published configuration | Nanbeige4.3-33B | Qwen3.8-27B text decoder |
|---|---|---|
| Decoder layers | 52 | 64 |
| Full-attention layers | 52 | 16 |
| Recurrent linear-attention layers | 0 | 48 |
Nanbeige's configured stack is uniform: each of its 52 decoder layers uses full attention. Qwen's text decoder repeats three recurrent linear-attention layers followed by one full-attention layer. That four-layer pattern gives it 48 recurrent and 16 full-attention layers across 64 layers. The map lets you select any layer to see its configured heads and widths.
The context slider at 32K tokens uses those pinned dimensions to calculate 6.98 GB versus 2.15 GB of raw f16 full-attention K/V tensors for one sequence if caching is enabled. This 3.25× ratio covers that component. Nanbeige's released model and generation config set use_cache: false. Qwen recurrent state, weights and runtime overhead are separate.
Why inspect this now? A model's parameter count does not tell you how its attention state is arranged. For long-context inference, the architecture suggests a useful measurement: compare latency and memory traffic across prompt lengths under a fixed runtime, quantization, KV-cache dtype and batch setup. Repeat the runs and report both prefill and decode separately. The layer map helps form that question; it is not a performance benchmark or a claim that one model is faster.
The figures above come from pinned publisher files: Nanbeige config at c1093c5 and Qwen config at 1d4bf0f. Qwen values refer to text_config inside the released configuration. Configured maximum positions are not measured usable context. We did not capture Nanbeige weights or activations for this comparison.
Explore the layer map in alphaBell Model Observatory. If you have matched runtime measurements for these models, share the quantization, context length, KV dtype and batch settings alongside the result.
