vxltxrllc/Qwen3-32B-BA-IQ4_NL

Qwen3-32B-Business-Agentic-GGUF

GGUF quant of Qwen3-32B, calibrated on agentic and business workflows, and also function calling.

Quantized using llama.cpp with a custom imatrix (importance matrix) in attempt to retain high logical reasoning and tool-calling accuracy even at lower bitrates.

Quantization Details

• Quant Types: IQ4_NL (~4 BPW)
• Calibration / imatrix: Calculated with high-context business & code logic dataset.

---

Usage Instructions

1. Running with llama-server / llama.cpp (ROCm / CUDA)

To enable tool calling and agentic capabilities, make sure to supply the --jinja flag:

llama-server \
  -m Qwen3-32B-BA-IQ4_NL.gguf \
  --jinja \
  -fa \
  -c 16384 \
  -ngl 99 \
  --port 8080

2. Running with Ollama

Create a Modelfile:

FROM ./Qwen3-32B-BA-IQ4_NL.gguf

PARAMETER stop "<|im_end|>"
PARAMETER stop "<|endoftext|>"
PARAMETER temperature 0.3
PARAMETER num_ctx 16384

TEMPLATE \"\"\"{{- if .Messages }}
{{- range .Messages }}
<|im_start|>{{ .Role }}
{{ .Content }}<|im_end|>
{{- end }}
{{- else }}
<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
{{- end }}\"\"\"

Build and run:

ollama create qwen3-32b-ba -f Modelfile
ollama run qwen3-32b-ba "Привет! Какие задачи ты умеешь решать?"