IST Austria Distributed Algorithms and Systems Lab

university

https://ist.ac.at/en/research/alistarh-group/

https://github.com/IST-DASLab

Activity Feed

AI & ML interests

None defined yet.

Recent Activity

anm2211 new activity about 4 hours ago

ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF:Updated chat template

anm2211 new activity about 7 hours ago

ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF:I think might have had one too many experts stripped out

helcig updated a model about 17 hours ago

ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF

View all activity

Papers

Hardware-Native Joint Sparse-Quantization for Trillion-Scale Mixture-of-Experts

WUSH-KV: KV Cache Quantization with Data-Adaptive Transforms

View all Papers

ISTA-DASLab 's collections 12

GSQ-RCO-GGUF

Non-uniform GGUF quantizations via GSQ + RCO: per-tensor mixed precision in standard GGUF form

ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF

Text Generation • 197B • Updated about 17 hours ago • 1.38M • 425
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF

Image-Text-to-Text • 177B • Updated 29 days ago • 7.44M • 1.08k
ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF

Image-Text-to-Text • 27B • Updated Sep 2 • 1.27M • 2.43k
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF

Image-Text-to-Text • 117B • Updated 9 days ago • 1.04M • 529

GSQ

GSQ: Highly-Accurate Low-Precision Scalar Quantization for LLMs via Gumbel-Softmax Sampling, https://huggingface.co/papers/2604.18556

GSQ: Highly-Accurate Low-Precision Scalar Quantization for LLMs via Gumbel-Softmax Sampling

Paper • 2604.18556 • Published Apr 20 • 24
ISTA-DASLab/Llama-3.1-70B-Instruct-2Bit-GSQ

Text Generation • 7B • Updated May 18 • 79 • 2
ISTA-DASLab/Llama-3.1-70B-Instruct-3Bit-GSQ

Text Generation • 9B • Updated May 18 • 74 • 1
ISTA-DASLab/Qwen3.6-35B-A3B-2Bit-GSQ

Image-Text-to-Text • 36B • Updated Aug 14 • 1.18k • 29

FP-Quant QAT

High-quality QAT FP4 models to use with the fp_quant vLLM/Transformers integration on Blackwell NVIDIA GPUs. See https://arxiv.org/abs/2509.23202

ISTA-DASLab/Llama-3.2-3B-Instruct-FPQuant-QAT-NVFP4

2B • Updated Oct 27, 2025 • 27 • 1
ISTA-DASLab/Llama-3.2-3B-Instruct-FPQuant-QAT-MXFP4

2B • Updated Oct 27, 2025 • 8
ISTA-DASLab/Llama-3.1-8B-Instruct-FPQuant-QAT-NVFP4

5B • Updated Oct 27, 2025 • 26
ISTA-DASLab/Llama-3.1-8B-Instruct-FPQuant-QAT-MXFP4

5B • Updated Oct 27, 2025 • 60

MR-GPTQ

MXFP4 and NVFP4 quantized models

ISTA-DASLab/Llama-3.1-8B-Instruct-MR-GPTQ-nvfp

Image-Text-to-Text • 5B • Updated Oct 3, 2025 • 31
ISTA-DASLab/Llama-3.1-8B-Instruct-MR-GPTQ-mxfp

Image-Text-to-Text • 5B • Updated Oct 3, 2025 • 30

QuEST

https://arxiv.org/abs/2502.05003

QuEST: Stable Training of LLMs with 1-Bit Weights and Activations

Paper • 2502.05003 • Published Feb 7, 2025 • 48
ISTA-DASLab/QuEST-800M-INT1

Updated Feb 18, 2025
ISTA-DASLab/QuEST-800M-sparse-INT4

Updated Feb 18, 2025
ISTA-DASLab/QuEST-800M-INT4

Text Generation • Updated Feb 19, 2025

AQLM+PV

Official AQLM quantizations for "PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression": https://arxiv.org/abs/2405.14852

PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression

Paper • 2405.14852 • Published May 23, 2024 • 2
ISTA-DASLab/Meta-Llama-3.1-70B-Instruct-AQLM-PV-2Bit-1x16

Text Generation • 11B • Updated Sep 17, 2024 • 33 • 45
ISTA-DASLab/Mistral-Nemo-Instruct-2407-AQLM-PV-2Bit-1x16-hf

3B • Updated Aug 16, 2024 • 15 • 3
ISTA-DASLab/Meta-Llama-3.1-8B-Instruct-AQLM-PV-2Bit-1x16-hf

Text Generation • 2B • Updated Aug 28, 2024 • 38 • 8

MoESQ

Paired-4:8 + NVFP4 W4A4 expert compressed MoE models for NVIDIA Blackwell Sparse Tensor Cores.

ISTA-DASLab/Kimi-K2.5-P48NVFP4-MoESQ

Text Generation • 646B • Updated 23 days ago • 658 • 1
ISTA-DASLab/Qwen3-30B-A3B-P48NVFP4-MoESQ

Text Generation • 20B • Updated 23 days ago • 959 • 6
ISTA-DASLab/Qwen3.5-397B-A17B-P48NVFP4-MoESQ

Image-Text-to-Text • 258B • Updated 23 days ago • 95 • 1
ISTA-DASLab/Qwen3.8-Flash-Next-P48NVFP4-MoESQ

Text Generation • 135B • Updated 23 days ago • 1.25k • 13

Optimizers

DASH: Faster Shampoo via Batched Block Preconditioning and Efficient Inverse-Root Solvers

Paper • 2602.02016 • Published Feb 2 • 14

MatGPTQ

MatGPTQ quantized models

ISTA-DASLab/Llama-3.1-8B-Instruct-MatGPTQ

Text Generation • 8B • Updated 7 days ago • 192
ISTA-DASLab/Qwen3-8B-MatGPTQ

Text Generation • 8B • Updated 7 days ago • 147 • 3
MatGPTQ: Accurate and Efficient Post-Training Matryoshka Quantization

Paper • 2602.03537 • Published Feb 3 • 6

Gemma3-GPTQ

GPTQ-quantized Gemma3 models

ISTA-DASLab/gemma-3-4b-it-GPTQ-4b-128g

Image-Text-to-Text • 4B • Updated Apr 13, 2025 • 495 • 7
ISTA-DASLab/gemma-3-12b-it-GPTQ-4b-128g

Image-Text-to-Text • 12B • Updated Apr 13, 2025 • 2.26k • 6
ISTA-DASLab/gemma-3-27b-it-GPTQ-4b-128g

Image-Text-to-Text • 28B • Updated Mar 20, 2025 • 409k • 44

HIGGS

Models prequantized with [HIGGS](https://arxiv.org/abs/2411.17525) zero-shot quantization. Requires the latest `transformers` to run.

Pushing the Limits of Large Language Model Quantization via the Linearity Theorem

Paper • 2411.17525 • Published Nov 26, 2024 • 6
ISTA-DASLab/Llama-3.1-8B-Instruct-HIGGS-GPTQ-4bit

Text Generation • 3B • Updated Dec 10, 2024 • 31
ISTA-DASLab/Llama-3.1-8B-Instruct-HIGGS-GPTQ-3bit

Text Generation • 2B • Updated Dec 10, 2024 • 34
ISTA-DASLab/Llama-3.1-8B-HIGGS-GPTQ-4bit

Text Generation • 3B • Updated Dec 10, 2024 • 29

AQLM

AQLM quantized LLMs

Extreme Compression of Large Language Models via Additive Quantization

Paper • 2401.06118 • Published Jan 11, 2024 • 15
ISTA-DASLab/Meta-Llama-3-70B-Instruct-AQLM-2Bit-1x16

Text Generation • 11B • Updated May 13, 2024 • 32 • 20
ISTA-DASLab/Meta-Llama-3-8B-Instruct-AQLM-2Bit-1x16

Text Generation • 2B • Updated Nov 8, 2024 • 96 • 12
ISTA-DASLab/Meta-Llama-3-8B-AQLM-2Bit-1x16

Text Generation • 2B • Updated Jul 2, 2024 • 32 • 8