🌌 Qwenius-4B-v0.05-imatrix-GGUF

An optimized fine-tune of Qwen 3.5 4B Base (Qwen/Qwen3.5-4B-Base), specialized in conversational reasoning, multi-turn dialogue, and complex logic puzzles.

This repository contains Importance Matrix (imatrix) quantized weights. Imatrix quantization uses calibration data to identify and protect critical weights during the compression process, resulting in lower perplexity loss compared to standard GGUF quantizations.

---

💾 Files in this Repository

• Qwenius-4B-v0.05-imatrix-q4_k_m.gguf (2.78 GB): The core 4-bit medium (Q4_K_M) imatrix quantized weights.
• mmproj-model-f16.gguf (672 MB): The companion 16-bit float visual projector required to process image inputs.
• .gitattributes: Standard git repository configuration file.

---

📚 Dataset Curation & Filtering Pipeline

To train Qwenius-4B-v0.05, raw conversational and logical reasoning traces were sourced from two datasets:

1. ianncity/GLM-5.2-Conversation: High-reasoning conversational traces distilled from GLM 5.2.
2. ianncity/GLM-5.2-Logic-Puzzles: Complex logical puzzles and step-by-step reasoning exercises.

🧹 Custom Preprocessing (Private Python Application)

Before initiating training, the raw data was filtered and cleaned using a custom-built, private Python application. Every training sample was constrained to a maximum length of 2,000 tokens. The filtering process pruned a large number of low-quality or out-of-scope samples to yield a finalized, high-quality training set of exactly 15,248 samples.

The curation rules implemented in the custom Python application were structured as follows:

1. Quality Filters & Denoising (Sequence Thresholds)

• Sequence Length Thresholds (Characters):
  • User Prompt: Min 1 / Max 500 characters.
  • Reasoning Block (Chain-of-Thought): Min 1 / Max 6,000 characters.
  • Final Output (excluding thinking tags): Min 1 / Max 1,500 characters.
• Turn Validation & Cleaning:
  • Enforce Turn Alternation: Ensured conversations strictly alternate between User ⇄ Assistant roles.
  • Filter Repetitive Generations: Repetitive loop patterns and degenerate output sequences were cleaned.
  • Require Reasoning Tag Pair: Only samples containing matching, unbroken thinking tag pairs were accepted.
  • Filter HTML & Unicode Noise: Unrendered HTML templates and binary/corrupted Unicode structures were stripped.
  • Filter Common AI Refusals: Systematic filtering was applied to prevent the model from learning repetitive refusal templates. This stripped out standard boilerplate refusal phrases such as:

    "as an ai", "as a language model", "as an assistant", "against my programming", "violates my guidelines", "i am unable", "i cannot", "i apologize but", "i'm sorry", "not ethical", "is illegal and unethical", "potentially harmful", "i'm not permitted to", "is not capable of" (among other variations).

2. Cross-Dataset Prompt De-duplication

Prompt comparisons were evaluated both within each individual dataset and across the entire dataset pool:

• Near-Duplicate Pruning: Enabled.
• De-duplication Algorithm: Jaccard (Fast and scalable token-set based approach).
• Similarity Threshold: 0.85 (A higher threshold requiring highly similar matches before removal, protecting unique semantic variety).
• Detailed Debug Logging: Enabled.

---

⚙️ Hyperparameters & Training Configuration

Fine-tuning was conducted locally via the Unsloth Studio Web UI with the following parameters:

🎯 LoRA Settings:

• Rank (R): 32
• Alpha ($\alpha$): 64
• Dropout: 0.00
• Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj

🔧 Optimization & Schedule:

• Optimizer: AdamW 8-bit
• LR Scheduler: Cosine
• Learning Rate: 0.0002 (2e-4)
• Batch Size: 2
• Gradient Accumulation Steps: 4
• Weight Decay: 0.0001
• Context Length: 2048

---

📈 Training Performance & Observability

The training run successfully completed with a steady, stable convergence.

⏱️ Training Statistics:

• Epochs: 1.00 (100% complete)
• Total Steps: 1890 / 1890
• Elapsed Time: 5h 20m 50s
• Training Speed: 0.10 steps/s
• Total Trained Tokens: 17,653,060 (~17.65M tokens)

📊 End-of-Run Metrics:

• Smoothed Loss: 0.5593 (Final Step Loss: 0.5594) 📉
• Learning Rate: Decay completed to 0.00e+0
• Gradient Norm: 0.287
• Evaluation Loss: Consistently decreased from 0.6068 (at step 100) down to approximately 0.55 (at step 1800), verifying solid generalization without overfitting.

---

💻 Hardware Setup

The model was fine-tuned locally using the following system hardware:

• GPU: NVIDIA GeForce RTX 5070 Ti (16 GB GDDR7 VRAM)
• System Memory: 32 GB RAM
• Export Path: Saved locally to C:\Users\Mat\.unsloth\studio\outputs\Qwen_Qwen3.5-4B-Base__project-qwenius-4b-v0.05_1784315694 before Hugging Face deployment.

---

🔧 Recommended Inference Parameters

For optimal generation behavior, reasoning stability, and coherent multi-turn outputs, we recommend using the following sampling parameters:

• Temperature: 0.6 🌡️
• Top P Sampling: 0.95 (Enabled)
• Top K Sampling: 20 (Enabled)
• Repeat Penalty: 1.0 (Enabled / Neutral)
• Presence Penalty: 1.3 (Enabled) ⚠️ (Crucial for loop mitigation)
• Min P Sampling: 0.0 (Enabled / Neutral)

⚠️ Critical Warning: Infinite Chain-of-Thought (CoT) Loops

Due to intensive training on complex reasoning traces, this model has a strong inclination to think deeply. However, under certain prompting conditions or with specific context windows, it can occasionally get stuck in an infinite chain-of-thought (CoT) loop (generating reasoning tokens inside its thinking tags indefinitely without transitioning to a final answer).

Mitigation Strategies:

1. Presence Penalty: Ensure that the Presence Penalty is set to 1.3 (as recommended in the configuration). This penalizes the exact repetition of tokens and structures, helping the model break out of reasoning loops.
2. Structural Stop Tokens: Set explicit stop sequences in your local inference client (such as Llama.cpp or LM Studio) to trigger on closing tags (e.g., </thought> or conversational turn-ending markers) to forcefully stop generations that run too long.
3. Entropy Insertion: If the model gets persistently stuck, slightly raising the Temperature (e.g., to 0.7 or 0.8) or increasing Top K can introduce enough entropy to derail a repetitive generation loop.

⚠️ Important Notice: Disable MTP (Multi-Token Prediction)

MTP speculative decoding must be kept disabled during inference for this model.

1. Architecture Limitation: Since this GGUF was merged from a custom QLoRA fine-tune, the model weights do not contain the specialized Multi-Token Prediction (MTP) output heads.
2. Vision Projector Conflicts: Multimodal inference utilizing the visual projector (mmproj-model-f16.gguf) does not natively support or compute with active MTP speculative decoding across many popular engines.

Ensure that speculative decoding / MTP features (such as --spec-type draft-mtp in llama.cpp or the "MTP Speculative Decoding" toggle in LM Studio) are switched OFF when loading this model.

---

🛠️ How to Use (Inference Guide)

Since this model format is GGUF, it can be run locally using lightweight engines like Llama.cpp, LM Studio, or Ollama.

🖼️ Multimodal (Vision) Inference:

Because Qwenius-4B-v0.05 is a vision-language model, you must load both the core model weights and the visual projector file (mmproj) to analyze images:

Using with Llama.cpp CLI:

To interact with the model alongside an image input, run:

./llama-cli -m Qwenius-4B-v0.05-imatrix-q4_k_m.gguf \
            --mmproj mmproj-model-f16.gguf \
            -p "Describe this image in detail and solve any puzzles visible in it." \
            --image path/to/your_image.jpg